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ABSTRACT 


Military aviation is the frontier of implementing leading edge technology. 
The major objectives of advanced technology aircraft are to increase pilot safety 
and mission efficiency; the Joint Strike Fighter, the most modern fighter aircraft 
under development, has many technological innovations for just this purpose. 

A common fact is that technology develops and is used faster than it can 
be researched thoroughly. This thesis seeks to identify and mitigate potential 
human factors concerns related to the Joint Strike Fighter, before it is used in the 
air forces of participating countries. The objective is neither to blame nor defend 
the design of the aircraft. 

Two surveys and an interview yielded the following findings: fighter pilots 
will use automation more in JSF than in their current types, the main LCD 
management will be key to mission efficiency and safety, the Distributed Aperture 
System should be addressed very carefully to avoid disorientation issues, and 
tactical decision-making skills will be more important and demanding. 

New approaches for better automation training, more focus on data 
filtering, display management, prioritization skills, establishing robust standard 
operating procedures for DAS, and addressing the complex decision-making 
skills in more detail than the current training curriculums are concluded to be the 
major requirements of JSF pilot training. 


V 



THIS PAGE INTENTIONALLY LEFT BLANK 


VI 



TABLE OF CONTENTS 


I. INTRODUCTION.1 

A. PROBLEM STATEMENT.1 

B. MOTIVATION.2 

C. OBJECTIVES.3 

D. THESIS ORGANIZATION.3 

II. RELATED WORK AND BACKGROUND.5 

A. COCKPIT AUTOMATION.6 

1. Introduction to Cockpit Automation.6 

2. Definitions and Overview of the Automation Probiems.9 

a. Mode Awareness/Confusion/Errors . 9 

b. Decompensation Incident, Automation Omission 

and Commission Errors . 12 

c. Clumsy Automation . 13 

3. Findings About Cockpit Design.14 

4. Findings About Piiots’ Strategies of Automation 

Management.19 

B. ALLOCATION OF ATTENTION, WORKLOAD AND TASK 

MANAGEMENT.22 

1. Attention Theory.22 

2. Attentionai Resources Considerations in Automated 

Tasks.24 

3. SA, Workioad and Task Management.25 

4. Auditory and Visuai Resource Considerations.35 

5. Decision Making Considerations.42 

6. Modeis to Predict Aiiocation of Attention.43 

7. FAA Research about Fiight Training.44 

C. THE IMPORTANT FEATURES OF JSF RELEVANT TO HUMAN 

FACTORS.49 

1. General Features and Systems of JSF.49 

2. Radar.51 

3. Electro-optical Targeting System (EOTS).52 

4. Distributed Aperture System (DAS).52 

5. Communication/Navigation/Identification (CNI) and 

Other Capabiiities.54 

D. POTENTIAL RESEARCH CONCERNS ABOUT HUMAN 

FACTORS IN JSF.54 

III. METHODOLOGY.57 

A. INITIAL SURVEY AND INTERVIEWS.57 

1. Initial Plan.58 

a. Negative Transfer of Training . 58 

b. Task Analysis . 59 

vii 





































c. Experiment . 60 

d. Security Ciassification and Required Literature . 60 

e. The Probiems Experienced . 61 

2. Preliminary Survey.62 

3. Interviews.65 

4. The Unclassified Simulator.65 

B. FOLLOW-ON SURVEY.66 

1. General Information.66 

a. The Structure of the Foiiow-on Survey . 66 

b. Participants . 72 

IV. RESULTS AND DISCUSSION.75 

A. PRELIMINARY SURVEY WITH INTERVIEW RESULTS.75 

1. Findings from Interviews, Preliminary Survey and Own 

Simulator Experience.75 

a. Summary of Preiiminary Survey Findings .75 

B. ANALYSIS OF THE FOLLOW-ON SURVEY RESULTS.81 

1. Multiple Choice Questions.81 

2. Open-ended Questions.99 

C. DISCUSSION OF THE FINDINGS.101 

1. Cockpit Automation.102 

2. Tactical Decision Making and Systems Management.103 

3. Transition Training.106 

V. CONCLUSIONS AND RECOMMENDATIONS.111 

A. SUMMARY OF KEY FINDINGS.111 

1. Increasing Use of Autopilot.Ill 

2. HMD and DAS Usage.112 

3. Display Suite Management.112 

4. Tactical Experience and Decision Making.112 

B. TRAINING OBJECTIVES.113 

1. Increasing Use of Autopilot.113 

2. HMD and DAS Usage.113 

3. Display Suite Management.114 

4. Tactical Experience and Decision Making.115 

C. GENERAL TRAINING GUIDE.116 

1. Increasing Use of Autopilot.116 

2. HMD and DAS Usage.120 

3. Display Suite Management.123 

4. Tactical Experience and Decision Making.125 

D. CONCLUSIONS.127 

E. FUTURE RESEARCH RECOMMENDATIONS.127 

1. Problem Identification.127 

a. Research Recommendations . 128 

2. Objective Methods.128 

a. Recommendations for Experimentai Research . 129 


viii 














































LIST OF REFERENCES.133 

INITIAL DISTRIBUTION LIST.139 


IX 





THIS PAGE INTENTIONALLY LEFT BLANK 


X 



LIST OF FIGURES 


Figure 1. The parts of automation processing (From: Wickens, 2000, p. 2).24 

Figure 2. Reported task prioritization incidents between given time periods 

(From; Funk, Suroteguh, Wilson, and Lyall, 1998, p. 4).28 

Figure 3. Factors affecting task prioritization (From: Funk et al., 1998, p. 6) 29 

Figure 4. JSF cockpit with LCD display suite (From: briefing received from 

PVI Team at Lockheed Martin, Fort Worth, TX).50 

Figure 5. Answers to Question 1.82 

Figure 6. Answers to Question 2.82 

Figure 7. Answers to Question 3.83 

Figure 8. Answers to Question 4.83 

Figure 9. Answers to Question 5.84 

Figure 10. Answers to Question 6.84 

Figure 11. Answers to Question 7.85 

Figure 12. Answers to Question 8.85 

Figure 13. Answers to Question 9.86 

Figure 14. Answers to Question 10.86 

Figure 15. Answers to Question 11 .87 

Figure 16. Answers to Question 12.87 

Figure 17. Answers to Question 13.88 

Figure 18. Answers to Question 14.88 

Figure 19. Answers to Question 15.89 

Figure 20. Answers to Question 16.89 

Figure 21. Answers to Question 17.90 

Figure 22. Answers to Question 18.90 

Figure 23. Answers to Question 19.91 

Figure 24. Answers to Question 20.91 

Figure 25. Answers to Question 21 .92 

Figure 26. Answers to Question 22.92 

Figure 27. Answers to Question 23.93 

Figure 28. Answers to Question 24.93 

Figure 29. Answers to Question 25.94 

Figure 30. Answers to Question 26.94 

Figure 31. Answers to Question 27.95 

Figure 32. Answers to Question 28.95 

Figure 33. Answers to Question 29.96 

Figure 34. Answers to Question 30.96 

Figure 35. Answers to Question 31 .97 

Figure 36. Answers to Question 32.97 

Figure 37. Answers to Question 33.98 

Figure 38. Answers to Question 34.98 

Figure 39. Answers to Question 35.99 


XI 










































THIS PAGE INTENTIONALLY LEFT BLANK 


XII 



LIST OF ABBREVIATIONS AND ACRONYMS 


A/A 

Air to Air 

A/G 

Air to Ground 

AIM 

Aeronautical Information Manual 

ANCS 

Aviate Navigate Communicate and System 

Management 

AO PA 

Aircraft Owners and Pilots Association 

ASRS 

Aviation Safety Reporting System 

BVR 

Beyond Visual Range 

CNI 

Communication/Navigation/Identification 

COMAO 

Composite Air Operation 

CRM 

Crew/Cockpit Resource Management 

DAS 

Distributed Aperture System 

DEAD 

Destruction of Enemy Air Defenses 

EOTS 

Electro-optical Targeting System 

FLCS 

Flight Control System 

FLIR 

Forward Looking Infrared Radar 

FPM 

Feet Per Minute 

GPS 

Global Positioning System 

HMD 

Helmet Mounted Display 

HOTAS 

Hands On Throttle and Stick 

HUD 

Head Up Display 

IR 

Infrared 

IRSTS 

Infrared Search and Tracking System 

JSF 

Joint Strike Fighter 

LAX 

Los Angeles International Airport 

LCD 

Liquid Crystal Display 

LTM 

Long-term Memory 

MET 

Maneuver Based Training 


xiii 



MFD 

Multi-functional Display 

NTT 

Negative Transfer of Training 

PEL 

Problem Eased Learning 

PVI 

Pilot Vehicle Interface 

RPD 

Recognition-primed Decision-making 

RWR 

Radar Warning Receiver 

SA 

Situational Awareness 

SET 

Scenario Eased Training 

SEAD 

Suppression of Enemy Air Defenses 

SEEV 

Salience/Effort/Expectancy/Valuable (A model of 

attentional allocation) 

SFO 

San Francisco International Airport 

SOP 

Standard Operating Procedures 

TAA 

Technically Advanced Aircraft 

TACAN 

Tactical Air Navigation 

VOR 

VHF Omni-directional Range 

WM 

Working Memory 


XIV 



EXECUTIVE SUMMARY 


Aviation is the frontier of implementing leading edge technology. Modern 
systems, displays and interfaces continuously enter cockpits to increase mission 
capabilities as well as safety and efficiency. However, it is up to the pilots to meet 
the promised objectives using the new systems. 

The military requirements and budget concerns during the design and 
acquisition phases generally limit the conduct of relative research on human 
factors issues about the new systems; thus, many issues are still being 
addressed long after the acceptance of the assets or their subsystems. In some 
cases the drawbacks result in mishaps, and in others less efficiency than 
expected, until the human factors problems are solved completely. 

Military aviation has the most demanding requirements among the aviation 
communities. The nature of the missions, the environment, and all additional 
stressors are the reasons for this fact; added to those factors is the need of 
military pilots to build more advanced skills than their civilian counterparts. The 
major task in commercial aviation is navigation, whereas navigation is only a tool 
to reach and return from mission areas in military aviation. Additionally, military 
pilots also accomplish air combat and night assaults out of many other military 
mission types. In order to succeed in their tasks, the military pilots use various 
enhanced systems that the commercial pilots don’t. 

Currently the Joint Strike Fighter is the most advanced fighter aircraft 
under development. Many allied countries contribute to the JSF project, and it is 
predicted that it will be the main asset of the NATO air forces in the coming 
decades. JSF has many innovations that will have impact on pilot performance. 
The increased use of automation (causing a role shift for the pilot), a big LCD 
display suite as the main PVI, a clear cockpit design with far fewer controls and 
switches, enhanced systems such as Distributed Aperture System (which allows 
pilots to “see through” the aircraft fuselage), and data fusion from many sensors 
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are among the main characteristics of the JSF cockpit. All of the aforementioned 
features are expected to cause some human factors concerns that must be 
researched and analyzed for safer and more efficient utilization. 

The literature includes extensive research on automation, workload, 
attentional management, prioritization and the effects of many systems such as 
HUD and HMD on human performance. The objective of this thesis is twofold: to 
investigate whether there are areas prone to those same problems found in the 
literature, and to anticipate the problems unique to JSF itself. Other than trying to 
identify potential problems, the second objective is to propose solutions in the 
training phase that will mitigate the predicted issues. 

The initial strategy of these authors was to investigate the potential 
problems regarding the transition phase. The JSF pilots will be chosen among 
the current types, and the main predicted concerns evaluated as they relate to 
the potential Negative Transfer of Training (NTT) issues during transition phase. 
The experiences of the pilots in their current aircraft types would make them 
prone to errors in JSF, and this possibility may result in mishaps or decreased 
efficiency. It was deemed possible and a major concern that the locations of the 
JSF cockpit controls and their operating procedures would conflict with the pilots’ 
prior experience. 

In order to identify the crucial areas of potential NTT and initiate the 
possible solution building process, the authors traveled to the Lockheed Martin 
facilities in Fort Worth, TX. The trip provided some opportunities to investigate 
the subject. These authors were able to experience the unclassified simulator, 
conduct a preliminary survey with fighter pilots who were experiencing various 
missions in the full mission simulator, and interview those pilots about their 
opinions. While expecting to return with all required information to complete the 
thesis with solutions, the authors found that the major problematic areas were 
quite probably different than those predicted as NTT during transition period. The 
participants and the authors mostly agreed that NTT will not be an important 
issue, but that role change, new systems and Pilot Vehicle Interface will be the 



crucial components about the potential concerns. Thus these authors decided to 
investigate the problem identification process deeper; otherwise the solutions 
would not address the actual issues. 

A follow-on survey was formed with multiple-choice questions in Likert 
scale, as well as open-ended questions to cover any missed issue in multiple- 
choice questions. The objectives were to identify the potential concerns robustly, 
and take opinions about the solutions. The following areas emerged as potential 
concerns: automation is predicted to be used in JSF much more than in current 
types, the Distributed Aperture System (DAS) is predicted to have some 
concerns if not addressed properly during SOP building and training periods, and 
the big LCD display suite management and tactical decision-making is predicted 
to be crucial for mission safety and efficiency. To adapt to the role change 
properly and build the requisite cognitive skills to operate the enhanced systems 
in JSF (with its different PVI) are thought to be the main challenges to training 
and following JSF pilots throughout their careers. 

The literature is full of studies about problems in cockpit automation, but 
mostly in commercial aircraft. The JSF is predicted to be the first fighter aircraft 
with considerable automation use; thus the findings of the literature can be 
tailored to confront any problems about the automation in JSF. Similarly, 
decision-making skills, workload, attentional resource management and 
prioritization are well studied, and the related findings from these studies can be 
also used for JSF as required. The only predicted problem that has not been 
researched directly is the use of DAS, because it has never before been installed 
in cockpits. 

A better approach to problem identification and building solutions could be 
utilizing objective methods such as experiments. But the security issues about 
the JSF project excluded that option in this thesis, and the literature and 
background of the participants as well as authors were the basis for the solution 
recommendations. 


xvii 



This thesis follows a bottom-up approach to generating training 
recommendations; the potential human factors concerns are predicted by the 
surveys, training objectives were established to mitigate the potential problematic 
concerns, and both findings of literature and operational background were 
utilized in order to recommend ways to address the objectives during the SOP 
building and training periods. 

The research literature review, subjective reports by the participants and 
operational experiences of these authors formed the basis for the 
recommendations to pilot training in JSF. A revision of the traditional training 
curriculums is recommended to address both the potential automation-related 
problems and to ensure that the pilots build relevant cognitive skills, while 
utilizing the ground training devices as much as possible. 
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I. INTRODUCTION 


A. PROBLEM STATEMENT 

Aviation is one of the leading application areas of technology 
development. Either it is the source of inventions itself, or implements already 
developed technologies faster than most of the other communities. These 
technological advances are reflected in cockpits mostly as increasing 
automation, new flight displays, and advanced avionics. The increasing number 
of sensors provides much more information to the pilots than ever before. Before, 
pilots were required to handle the throttle, stick, and a few manual systems. 
However, with the development of technology, the systems become more 
complex and have more sensors with greater range. These systems used in 
modern cockpits include radar. Radar Warning Receiver (RWR), jammers, 
weapon and flight management systems, etc. The implementation and upgrade 
of these systems raise the issue of their management, which resulted in the 
evolution of the conventional throttle and stick into the more complicated Hands 
On Throttle and Stick (HOTAS) system. 

The dramatic increase in mission effectiveness is undeniable. However, 
these improvements in systems were associated with human factors issues, 
besides their benefits. The fundamental task of pilots is still the same—to fly 
safely, to evaluate the displays and information presented on them, and to use 
this information effectively to make the best available decision. One could 
wonder where the human factors issues arise from if the task of the pilot is still, in 
essence, the same. The best way to answer this question would be to consider 
the differences in pilot-vehicle interfaces (PVI). Earlier, there were separate 
displays for each particular source of information, and the pilot switched his 
attention between displays frequently to continuously track and evaluate his 
situation in the tactical arena. On the other hand, most of the flight instruments 
used in cockpits were manual and analog. Recently, with the advance of Liquid 
Crystal Display (LCD) technology and sensor technology, engineers could 
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produce more effective ways of information presentation, not to mention the 
ability to gather more and various kinds of information that were not considered 
before. 

The pilots have access to much more information, but filtering the 
information to suit the specific tasks became an issue, as is managing the 
displays. Automation itself also caused some new problems for pilots. On top of 
all these advancements, generation fighters, particularly the Joint Strike 
Fighter (JSF) bring even more modern and new systems to life. Increased sensor 
and data fusion and the Distributed Aperture System (DAS) (which enables pilots 
to “look through” the fuselage in a full 360-degree arc) are among those modern 
systems. It also has more sophisticated automation as well as a relatively clear 
cockpit with fewer controls compared to older aircraft. 

There is a need to identify and understand the Fluman Factors concerns of 
the Joint Strike Fighter clearly and robustly before addressing them in proper 
ways. Some of the questions are “How will the pilots filter the data and focus only 
the required ones?”, “What kind of new skills will be needed for new generation 
fighters,” “Which areas of JSF are going to cause problems to the pilots, and thus 
need to be addressed carefully during training?”, or “What kind of concerns can 
be predicted about pilot-autopilot interaction in JSF?” Identifying the problems 
and important concerns and addressing them properly will allow pilots to benefit 
from the new assets more safely and efficiently. 

B. MOTIVATION 

When examining the human factors in aviation, one can find out that most 
of the regulations are based on previous accidents and mishaps. Adaptation to 
newly exposed systems, technologies and tactics brings previously unknown 
issues that generally result in safety problems, accidents and even loss of 
aircrew. The majority of these problems in the past can be seen as negative 
transfer of training in the transition phase, because the systems operated with 
more or less the same logic only with different displays and controls. Therefore, 
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previous experiences resulted in some safety problems due to new cockpit 
controls. However, the issue now is different. Advanced technology has offered 
new systems that are dramatically different from their predecessors, both in 
operating logic and pilot interface. The motivation of this thesis research is to 
anticipate the possible human factors issues beforehand, which might help to 
avoid possible accidents and mishaps, as well as increasing the effective use of 
systems. 

C. OBJECTIVES 

This thesis attempts to provide a comprehensive insight into the human 
factors issues of JSF. The main objective is to identify the important and 
problematic human factors concerns of JSF. After the identification of the 
problems, it will be possible to address them and propose solutions to those 
areas in order to increase the safety and effectiveness during the transition 
periods and thereafter. One way of proposing solutions is to use the literature 
and previous knowledge, and the other way is to address them with specific 
experiments, or field studies. The last and undesirable way is to learn them via 
experience, and the intent of this thesis is to identify and address them before 
any hazardous situations happen. 

D. THESIS ORGANIZATION 

1. Introduction. This chapter gives a general outline of the work and 
defines the problem the authors are trying to solve. 

2. Related Work and Background. This chapter discusses the primary 
concerns related to pilot workload, cockpit automation and attentional 
resources theory and summarizes literature reviews on these fields. 
The capabilities and key features of JSF are also discussed in this 
chapter. 
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3. Methodology. This thesis followed survey methodology for research, 
and conducted two surveys and interviews. This chapter explains the 
background ideas and structure of the preliminary and follow-on 
surveys, as well as the interviews. 

4. Results and Discussion. The results of the interviews and surveys are 
provided and discussed in this chapter. 

5. Conclusion and Recommendations. After reviewing the key findings of 
the surveys and interviews, the establishment of relevant training 
objectives and how to address them in pilot training topics are 
discussed in this chapter. Additionally, recommendations for future 
research are also included. 
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II. RELATED WORK AND BACKGROUND 


With evolving technology and growing experience in all scientific fields, the 
literature expands continuously. This sometimes creates new fields; at other 
times it enriches the content of an existing one. The main consequence of this 
phenomenon is the increase of required expertise in almost every research, 
project, or field of technology. 

The scope of this thesis is the Human Factors concerns in the JSF 
cockpit. As mentioned above, even this scope entails having information about 
many Human Factors subfields. Many researchers have studied modern glass 
cockpits, and two major topics have emerged from the literature about the 
Human Factors in such an environment: cockpit automation, and pilot workload 
along with management of attentional resources. 

Cockpit automation is a well-studied topic in commercial aviation, but is 
relatively fresh for military aviation, especially for fighter cockpits. Today, almost 
all commercial airplanes have very sophisticated automation systems, and as 
Olson (2001) reported, they are able to perform all tasks but takeoff and landing 
automatically. It is legitimate to say that even takeoff and landing can be 
performed with newer automation capabilities. Automation has also brought 
some problems along with its solutions, and all of those have been robustly 
researched in the literature. 

The other accepted truth is the increasing workload for the pilots. Pilots 
must learn and use many more systems than before, and the structure of the 
systems is more complex. All these facts, with the increasing demands of the 
mission environment, result in increased pilot workload in the cockpit. Both the 
designer and operational communities strive to build suitable systems and 
operational procedures, but there are many causes for concern. Sometimes 
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technology develops faster than humans can adapt, thus adding work rather than 
subtracting it. Workload is a well-known and studied topic by the human factors 
and ergonomics experts. 

With the increased level of automation, and many more sophisticated and 
revolutionary systems, the JSF will also require focus on these topics. For this 
reason, the related work and background section of this thesis is divided into two 
subsections, including cockpit automation and pilot workload. 

A. COCKPIT AUTOMATION 

The role shift of the pilots was one of the objectives of the Lockheed 
Martin team; Kent (2006) quoted Skaff’s statement that, “The F-35 cockpit design 
is driven by the desire to return the pilot to the role of tactician” (para. 12). This 
goal alone makes it worthwhile to study findings about automation for developing 
JSF training. This chapter reviews the findings of research literature related to 
cockpit automation. 

1. Introduction to Cockpit Automation 

Before investigating the human factors concerns, safety and effectiveness 
aspects of the cockpit automation thoroughly, one has to understand the reasons 
for integrating automation with the cockpits. 

Sarter, Woods and Billings (1997) listed improved economical efficiency 
and precision as two of the general benefits of automation. Flowever, they also 
add that these benefits have introduced other problems, and the importance of 
the human user has been reinforced, even in the highly automated cockpits. 
They claim that more training should be added to the curricula to address the 
automation-related problems in order to reduce the costs of the benefits. 

Another obvious reason is the increased workload in the modern glass 
cockpit. Most modern cockpits contain many more systems along with more 
sophisticated menus and operational procedures. In some situations, performing 
all of the required tasks is beyond users’ capabilities. For instance, most modern 

fighter aircraft have Flight Control Systems containing certain safety limitations; 
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they do not allow pilots to exceed structural g limits. The designers are aware of 
the workload and stress levels during air combat, and because there is a great 
possibility that the pilot may forget to monitor the g levels in such situations, they 
automated the FLCS to limit the pilots instead. 

After the overview of the main reasons for automation in cockpits, it can 
be seen that increased automation is inevitable for future platforms. There is no 
strong debate on the necessity of automated systems, but naturally it has been 
changing the pilots’ tasks and workload. A rough comparison would be a shift 
from performing fewer and simpler tasks alone to performing more and more 
complicated tasks together with automated systems. The automated systems 
can be the autopilot navigating the airplane and the radar system performing 
locking and tracking tasks, freeing the pilot for other imperatives. 

Sarter, Woods and Billings (1997) said of the common role change in the 
modern cockpits that, “Introduction of new automation has shifted the human role 
to one of monitor, exception handler, and manager of automated resources” (p. 
2). The same role shift is also acknowledged in the military aviation community, 
and Olson (2001) is one of the researchers who agreed that the shift in the role 
of pilots with increased automation will be from performing all the tasks to 
supervisory control. 

The role shift in the cockpit requires pilots to perform their tasks in a 
different way than before. Furthermore, they have to develop the corresponding 
skills and abilities to be successful in their new role. It is known that actually 
performing a task versus observing it (vigilance task) offers an improvement in 
terms of performance. It is a common fact that humans are not so effective in 
vigilance tasks. Rigner and Dekker (2000) acknowledged the effects of 
automation on required pilot skills. They said that the pilots have become 
supervisors in the cockpit, observing and maintaining the operation of all other 
systems and components. In order to succeed in their new roles, pilots should 
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absorb more knowledge and build many more complicated skills. Managing their 
attentional resources, and filtering and evaluating required data, are some of the 
important skills they need. 

An interesting point between Sarter, Woods and Billings (1997), and 
Rigner and Dekker (2000) is the need for cooperation with other systems or 
resources. Sarter, Woods and Billings (1997) confirmed the importance of the 
human-machine coordination, and acknowledged that the relationships and 
reliance factors are highly complicated in modern systems. The pilots should 
develop satisfactory mental models about the automated systems. There are 
many types of complicated relationships between the systems themselves and 
between the pilots and the automated systems. Pilots need to know and direct 
the systems interactions, and be aware of the all inputs to and outputs from the 
autopilot during the flight. 

Apparently, the skills emerging with the use of automation are the 
supervisory skill to observe the automated systems and the coordination and 
cooperation skills to work with the automated systems as if they were another 
agent in the cockpit. There are many instances proving that the implementation 
of automation in the cockpits can be problematic. Olson (2001) reported one of 
the important findings about the pilot-autopilot coordination: in most cases, after 
the automated system starts to perform a task, the pilots tend to forget to control 
the related performance parameters. This finding directly shows that the pilots do 
not observe the automation performance as they should. Initially, one can directly 
blame pilots, but this would be not the real solution, but an escape from the big 
picture. The problems should be addressed beginning with the design of the 
system, building the operational procedures, and training to control quality 
throughout the lifetime. The common problems of automation will be discussed in 
the next chapter, and the recommendations in the last one. 

To provide a quick view of the big picture about relations in the automated 

cockpits, Spencer (2000) mentioned a beneficial model. The main idea of the 

model is to define the systems with the subcomponents of software, hardware, 
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environment and liveware. The liveware in the cockpit example is the pilot, and 
he interacts with all of the other components during the missions. The other 
components are: standard operational procedures, the software of the pilot 
vehicle interfaces, the aircraft and the related hardware, co-pilots, etc. 

2. Definitions and Overview of the Automation Probiems 

In order to understand and interpret the results of the research better, it is 
crucial to acquire enough knowledge about the terminology and common 
knowledge of related fields. This is also applicable to cockpit automation, about 
which there are several commonly accepted definitions. These definitions, along 
with some examples will be reviewed in this chapter to provide the fundamentals 
needed in the following sections. Although there exist many other phenomena 
about cockpit automation, the mentioned ones are the most important and 
recurring ones in the literature. 

a. Mode Awareness/Confusion/Errors 

The reason to mention these three terms together is the fact that 
they are inextricably inter-related. They are used interchangeably in many 
studies, and are based on the same phenomenon. 

Today, most of automated systems consist of many operating 

modes and menus. They are also capable of making many decisions and 

performing tasks independently. As mentioned before, it will be legitimate to think 

that there are two actors in a highly automated cockpit, and therefore a very 

logical issue is the coordination between them. In other words, pilots should be 

aware of what the other decision maker is doing at all times, and how it also 

interacts with the other systems. This straightforward logic is self-explanatory, 

and even if one does not have any knowledge about cockpit automation, he can 

conclude that “mode awareness” means continuously to be aware of the mode in 

which the system is operating. Similarly, if the operator is confused about the 

mode of the automation, it is “mode confusion;” and if he commits errors because 

of this confusion they are “mode errors.” Even though they seem very 
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straightforward, there are many valuable and important data about mode errors 
in the literature, and these will be discussed later in this study. 

Sarter (2000) is one of the researchers who has studied cockpit 
automation. She said that the pilots generally do not have a through 
understanding of the structure and operational procedures of automated 
systems, and when combined with “low observability,” this creates various 
problems in flight (p. 233). Yet another additional system is then implemented to 
help pilots, with yet greater need for additional visual and mental resources. 

The reasons for the mode awareness problems in Sarter’s study 
are the knowledge problems and low observability; furthermore, Sarter, Mumaw, 
and Wickens (2007) defined four major factors causing the automation problems 
in glass cockpits. The first one is related to the lack of capturing the attention of 
the pilots. Most automation systems have poor design and fail to provide 
sufficient and easily observed feedback to pilots. Also, the increased number of 
automated actions shunt the pilots increasingly into a supervisory role; the 
automated systems initiate more actions without any pilot consent, which makes 
it harder to track the modes. Another reason is the “high degree of system 
coupling” (p. 348). In general, not only the modes and complexity of the 
automated systems, but also the number of other systems have increased. 
These developments conspired to hinder the pilots’ ability to track and 
understand the interactions between the automated and other systems. More 
systems became related to the inputs and outputs to automation. The last 
contributing factor is the knowledge of the pilots. It was found that the pilots’ 
lacked robust understanding of the automated systems, and were prone to mode 
awareness problems. 

When the operator takes an action as a result of thinking a system 
is in one mode, where it is actually in another mode, this is a “mode error” (Sarter 
et al., 2007, p. 347). 
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Examples will give a better insight on the mode awareness 
concerns. A mode can be a navigation mode of an autopilot system, or a tracking 
mode of a fighter’s fire control radar. In both situations, it is very crucial to be 
aware of the mode of the automated system. Most of the evidential accidents 
about the automation can be found in the literature. Sarter (2000) mentioned two 
of these incidents. In the first one, the captain changed the descent mode of the 
autopilot, because it deviated from the approach track, and switched to the 
“heading select mode.” Because the vertical operation part of the autopilot was 
working with the lateral part, the change resulted in an unintended 1800 fpm 
descent, which was considerably in excess of the proper rate for landing. 
Thankfully, the warning system in the aircraft and the crew of the control agency 
together prevented a potential accident. In the second example, the pilots 
thought that they were making a 3.3 degrees descent, whereas the autopilot 
commanded a 3300 fpm descent; the crew and passengers were not as lucky as 
in the first example, unfortunately, and the plane crashed. These real life 
examples indicate the importance of mode awareness in autopilot systems. 
Because the common approach is not to perceive the autopilot as a critical 
system, the aviation community must make sure to take the preventive 
measures. 

Although there are no examples about mode confusion in fighter 
jets, it does not mean that fighters will not have those problems. The main reason 
is the adoption of automation. Commercial aviation already implemented the 
automated systems, and most of the tasks can be performed with the autopilots. 
But to date, the fighter cockpits do not contain the same degree of automation. 
The autopilot and other automated systems are not used and implemented as in 
the commercial aviation community. But this trend is changing, as the authors 
observe in the JSF example. Not just the autopilot, but also even the auto-mode 
radar may cause problems with mode confusion. For instance, if the digit on one 
side of a radar target symbology represents the altitude in one mode, but another 
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data in another mode, the pilot may build an intercept strategy depending on the 
wrong interpretation of the data, which in turn may mean life or death. 

b. Decompensation Incident, Automation Omission and 
Commission Errors 

The “decompensation incident” occurs when automation tries to 
handle some problem or abnormal parameter up to a point, and then quits 
handling that problem. In such situations, pilots are liable to become aware of the 
problem too late (Sarter et al., 1997, p. 7). This definition is similar to the 
definition of the “automation omission error.” Mosier, Skitka, Heers, and Burdick 
(1998) explained the automation omission error as an error caused by a lack of 
correct information about an abnormal situation during automated operations (p. 
51). The example they give is an airline mishap in 1996. The airplane was 
controlled by the autopilot for level flight. One of the aircraft engines failed, but 
the autopilot compensated for the loss of thrust caused by that engine, and as 
the aircraft continued to fly level, the pilots did not understand the abnormal 
situation. Once they disengaged the autopilot, however, the airplane entered into 
an abnormal descent. 

Mosier et al. (1998) also explained the automation commission 
errors. These errors happen when pilots monitor contradicting behaviors between 
other and automated systems. The example they mention from the literature is a 
study with an experimental design. The pilots were given a scenario with 
contradicting cues. During takeoff, the automatic checklist directed the pilots to 
shutdown an engine, though the truth as shown by traditional displays was 
different. The other engine was malfunctioning, but 75% of the participants took 
improper action and trusted the automated system without cross checking it with 
traditional instruments. Both types of errors happen because of the lack of cross 
checking of other flight instruments with the automated ones. This finding, 
combined with the finding that pilots do not include status checks of automated 
systems in their routine checks (as they did with other systems and aircraft 
performance variables) indicate a problem in automation management. 
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c. Clumsy Automation 

One of the major promises of automation was reduced pilot 
workload—because the pilot would perform fewer tasks due to automation, the 
overall workload could be expected to decrease. This is a topic for debate, and 
Sarter et al. (1997) explained the definition of “clumsy automation” by Wiener 
(1989). They explained that “many automated systems support pilots most in 
traditionally low workload phases of flight but are of no use or even get in their 
way when help is needed most, namely in time-critical highly dynamic 
circumstances” (p. 3). This will quite probably be the case in the new fighter 
cockpit environment. The intent of the authors is not to say that the fighter 
cockpits have clumsy automation, but that the environment is generally as 
mentioned above: timely, critical and highly dynamic (Sarter et al., 1997). 
Another important change in the fighter cockpits is the nature of the missions. 
The new missions require many demanding and dynamic tasks such as air 
combat, SEAD, and DEAD missions, whereas the navigation is the only task in 
commercial aviation. This concern will also be discussed in the following 
chapters. 

In addition to the aforementioned problems, Doherty (2001) 
reported “boredom” and “complacency” (p. 22). Because the pilots feel that the 
reliance on them by the systems is decreased due to automation, their 
performance tends to decrease with the automation-related tasks. The effect of 
boredom is that it causes the stress level to be lower than the optimum, and the 
effect of complacency is the increased demand for cognitive sources. The 
boredom can cause a decrease in pilots’ SA, whereas the complacency can 
cause task saturation under highly dynamic and high workload situations. 

The main objective of this section was to provide the readers with 
fundamental knowledge about the human factors concerns about cockpit 
automation. The findings will be discussed further in the next sections. 
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3. Findings About Cockpit Design 

Inevitably, as more and more technological systems are used in the 
cockpit, many human factors concerns accompanied them. Automation is one of 
the major evolutions among the modern technological systems in glass cockpits. 
It has two aspects: hardware and software. In some instances, pre-existing 
systems have been digitized, and algorithms have been specifically coded to 
automate many tasks. In other cases, hardware is designed specifically to 
automate some tasks, such as autopilots. Regardless of the method, the process 
has had a major impact on human factors in cockpit design. 

Spencer (2000) explained the environment in modern glass cockpits as 
“Complex cockpits, faster, more capable aircraft, airspace saturation, and 
increasing air traffic control requirements create the environment and conditions 
conductive to mode confusion” (p. vi). 

The reason to include this chapter is neither to defend nor to criticize the 
design of the JSF cockpit, but to identify the potentially problematic areas. One of 
the commonly accepted impacts is the shift in the pilot’s role in the modern 
cockpit. This is a topic studied thoroughly under CRM and other automation 
research. Spencer’s model (described previously) for the interactions between 
pilot and components in the cockpit treats the pilot as essentially another system. 
This systems approach may provide designers a framework to consider the 
relations in the cockpit because, as he notes, the tendency with modern cockpit 
design is to prioritize the automated systems, whereas a better way is to use a 
human-centered approach (Spencer, 2000). As seen also in many other studies, 
the human has not been the major concern in modern cockpit design. However, 
the pilot is still responsible for safe and efficient operation of the aircraft. Some 
steps forward can be seen in many cockpit designs, but there is room for 
improvement on this issue. 

Likewise, another consequence of the automated cockpits is the shift in 
the type of activities conducted by pilots. They are not actively performing, but 
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observing the automated tasks, and it is commonly accepted that humans cannot 
maintain their performance during longer periods of vigilance tasks. Davies and 
Parasuraman (1982) are among the researchers who studied this phenomenon 
by questioning the findings’ relevance in real conditions. They explain the 
shortcomings of some findings of fully laboratory experiments, but also agree 
with the decreasing performance over time shown in many other studies. With 
the tasks the autopilot performs, and the pilot observes, there can be 
shortcomings due to vigilance. That raises the question of the number of tasks to 
be automated during design. Spencer (2000) reported the same issue, and noted 
that there is a threshold for the best performance in automated cockpits. 
Basically, up to this threshold more automation results in better performance. But 
after this point, if additional tasks are also automated, the pilot performance 
suffers due to SA problems. He added that “Researchers recommend balancing 
human involvement (increased situational awareness) with automation efficiency” 
(p. 12). This conclusion may be valid in general, but establishing a balance will 
be harder in dynamic situations. All of a sudden, additional threats may pop up, 
thus shifting the balance point for that situation. With these challenges in mind, 
the human involvement should at least be taken into consideration during design 
and operational phases. 

Since more systems are in use in modern cockpits, the users have to do 
more with their same cognitive resources. In general, mental resources are a 
concern for pilot workload studies, but the over-reliance on the visual resource is 
becoming a more important concern. The pilots monitor basic flight instruments, 
navigational and communicational systems. Air to Air and Air to Ground Radars, 
Radar Warning Receivers, automated systems and many more, and they use 
their visual system for all of these tasks. Increasingly, the question is whether the 
pilots’ visual resources are able to handle this workload in all of the required 
situations in the cockpit. In fact, the automated systems also demand visual 
resources in many applications, adding yet another system to help pilots but 
contributing to a need for additional visual resources. 
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Sarter (2000) tried to come up with solutions for decreasing the attention 
load on visual resources. In order to mitigate the problems with automation, one 
has to understand the reasons for the problems during automation use. Sarter 
(2000) reported one of the findings of the study by Sarter and Woods (1995), 
where the pilots did not implement the automated systems checks to their routine 
procedures, but mostly observed them depending on their “expectations of 
specific automation behavior” (Sarter, 2000, p. 234). She concluded that this 
approach will fail if the pilot is not fully aware of the status and modes of the 
autopilots (Sarter, 2000). The reason may not be the overload of visual 
resources, but additional resources can be addressed in order to provide pilots 
with increased awareness about the automated systems. 

Another issue Sarter (2000) noted is the problems in display designs of 
automated systems. The modern trend has been to convert the round shaped 
analog displays to “so-called tape instruments” (p. 236), but the pilots, at a 
glance were able to retrieve the required information from the round displays. 
She reported one of the findings in a study of Sarter and Woods (1995b), that 
pilots expressed difficulties about the non-round displays. They added that it took 
longer to get the data from tape displays. That finding points to the issue of 
additional cognitive processes and the increased demand on visual resources. 
This issue is important even if the automated systems do not have tape displays. 
The common use of tape displays is in flight performance parameters, and if the 
autopilot is controlling the airplane, those displays become one of the crosscheck 
displays to the pilots monitoring the automated systems. 

Another automation aspect of cockpit design is the reliance on the 
automated systems and their programmed algorithms. Automation is intended to 
provide safe and efficient operations, but there remains the risk of relying on the 
technology too much. Dalcher (2007) studied this issue by examining a case in 
his research where the autopilot limited the pilot to an extent that he couldn’t 
recover the aircraft; in that case, the automation itself caused the aircraft to 
crash. Dalcher believes that the goals with the highly automated systems are “... 
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the superior computational capability coupled with elimination of human error, 
and the reduction in work overload and lack of dependability,” and then 
acknowledges that there may be many situations where the automated systems 
lack the flexibility to handle difficult situations (p. 352). This is related to the 
algorithms and design of the automated systems. Many tasks are automated, but 
sometimes it may not be possible to consider all of the possible situations or 
inputs and outputs when designing the responses of the automated systems. 
Thus, the designers should be very cautious while choosing and implementing 
the tasks to be automated. 

After expressing the importance of involving users in the design and 
operational considerations, looking to the user side of automation design will 
provide valuable clues. Tenney, Rogers and Pew (1998) conducted a study via 
survey methodology, and tried to capture the pilots’ opinions about cockpit 
automation. The following findings are listed in their study. First, pilots’ 
preference was that the automation should be a cooperative design, rather than 
highly automated cockpits that make pilots mostly observers. Pilots want 
automation in their complex tasks, but they desire a human-centered approach, 
and cooperation among the agents in the cockpit. However, they reported some 
modes of automation as “...producing a high mental workload.” The pilots found 
the most important features to be “simplicity and reliability” (p. 103). The reliability 
is more related to design issues, and must be considered from the conceptual 
design phases to the end of the life cycle of the products. But it is not planned 
and implemented by the users. This is not the case in simplicity. Simplicity is a 
usability issue, and the user will be directly affected by it. Simple systems will 
decrease the workload of the operators, and free time for other tasks. 
Furthermore, users will be prone to fewer failures with easy-to-use systems. It is 
very apparent that the pilot opinions basically confirm the findings of the 
aforementioned studies, further validating those findings. 

The types of automation play a great role in assessing the attentional 
pattern of pilots and their mental approach to automation. Wickens (2000) 
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divided automation into two main groups: “reliable” and “unreliable.” He defines 
reliable automation as ’’that which does what it was intended to do” (p. 4), and 
suggests that reliable automation causes either “attentional tunneling” or 
“cognitive tunneling,” depending on the cognitive process. 

The same phenomenon is named by Wickens (2000) as “action tunneling” 
if it happens following cognitive tasks about automation, since the automation is 
naturally going to direct the pilot’s attention to the automation’s preferred choice 
of action. Wickens (2000) explains this term as an alternative action preferred by 
automation rather than the pilot, which, however, is not the best alternative at the 
moment because of considerations that are not related to automation. In this 
case, instead of devising an action himself, the pilot’s attention is going to be 
allocated to validating the automation’s recommended action. 

Supporting this suggestion, Wickens (2000) observed this phenomenon in 
an experiment in which pilots did the pre-flight check with wearable-computers. 
The experiment results showed that the detection of unidentified faults decreased 
while the detection rate of computer-identified faults increased. The information 
that is considered to be significant and is highlighted by the automation might 
cause the pilot to ignore other information that is not highlighted at the moment, 
but is as critical as the highlighted one. 

Eventually, says Wickens, the pilot will begin to trust in automation more 
than in himself; he will not question the validity of automation-inferred data and 
actions as much as he should. This will lead to the failure to detect an automation 
problem, degrade the level of situational awareness of the pilot, and cause him to 
lose his skills that have been replaced by automation. The loss of situational 
awareness due to the loss of automation will cause the pilot to evaluate the raw 
data and the systems outputs either slowly or not at all. In any case, the pilot will 
fail to consider all possible alternative actions due to this deprived situational 
awareness caused by “automation complacency.” 


18 



4. Findings About Piiots’ Strategies of Automation Management 

Sarter and Woods (1992) mention the commonly accepted perspective of 
Billings (1991), that the job of the pilots has shifted to that of a supervisor only 
actually performing in abnormal procedures; at the same time, they also discuss 
that taking the human user out of the loop is a risky decision that will affect the 
concentration and “awareness” of the pilots negatively. 

Sarter and Woods (1992) conducted a study in 1992. They report that 
even the pilots who used the automated systems more than one year sometimes 
had difficulties operating and understanding the automated systems, and they 
recommended that enough time should be given during pilot training to help them 
build a through understanding of the automated systems, and also to address the 
pilot-autopilot “coordination” during training. This was one of the earlier studies, 
and pointed out an important problem about the pilot-autopilot interaction. It is 
legitimate to think that the automated systems were not as complex then as they 
are today; thus, even with less complex systems offered the same problems. But 
another factor could be the fact that automation was in its earlier stages, and 
therefore that robust training and operational procedures had not been fully 
developed. So, reviewing further studies may make the subject more clear. 

Sarter and Woods (1994) also conducted an objective study, with a typical 
flight scenario from LAX to SFO with a part task trainer. They injected events 
related to automation during flight, and recorded pilots’ reactions to a prepared 
list. Using this method, they also asked participants questions about automated 
systems during low workload conditions. They tried to capture the pilots’ 
knowledge level in a realistic environment rather than in isolated situations. They 
report that pilots were comfortable in performing the standard basic tasks in 
cockpits such as “intercepting a radial, building or executing a holding pattern...,” 
but they experienced difficulties during the tasks requiring comprehensive 
understanding about the automation such as “aborting a takeoff at 40 knots with 
auto-throttles on“ and “...anticipating when go-around mode becomes armed 

through landing....,” both indicating that the pilots have insufficient knowledge 
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about the automated systems, even though they think that they have enough 
knowledge (p. 14). The interesting point is the lack of knowledge and operational 
capabilities about automation even in some emergency situations. Even if the 
autopilot may not seem complex and important, the mishaps and crashes 
express its importance. 

Based on the findings. Barter and Woods (1994) suggest improved 
training to address developing automation skills. The pilots not only have to 
understand the technical structure of the systems, but also be able to make 
decisions under high workload conditions. This requires a thorough 
understanding of the functions and interrelations of the automated systems. This 
is an issue to be addressed properly in realistic environments, via scenarios. The 
knowledge and operational abilities of pilots should be stressed with these 
scenarios in order to help them build proper mental models. 

Barter, Woods and Billings (1997) claim that newer approaches in 
automation training should not only include increasing the training time, but also 
modifying the nature of the content. The two important traps about automation 
management reported in their findings are the subjective opinion of the pilots that 
they have enough capabilities and knowledge, and their tendency to build only 
limited skills that they try to apply to all situations. These two points mask the 
truth for pilots, and in case of a life or death situation, it may be too late to 
discover their deficiencies in automation. 

Mosier, Bkitka, Heers, and Burdick (1998) investigated “automation bias” 
during automated tasks performed by commercial pilots. They conducted an 
experiment measuring the “automation omission,” and “automation commission” 
errors of the participants during automated tasks. For instance, commission 
errors happen when pilots monitor contradicting behaviors between traditional 
and automated systems. The example they give is when pilots were given a 
scenario with contradicting cues. During takeoff, the automatic checklist directed 
the pilots to shutdown an engine; the truth as shown in traditional displays was 
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different. The other engine was malfunctioning, but 75% of the participants took 
improper action and trusted the automated system without cross checking it with 
traditional instruments. 

Both omission and commission errors happen because of the lack of cross 
check of traditional flight instruments along with automated ones. This finding, 
combined with the finding that pilots do not include a routine status check of 
automated systems as they do with traditional systems and aircraft performance 
variables, again indicates the problem of automation management. Additionally 
another interesting finding of their study is: “...pilots who reported a higher 
internalized sense of accountability for their interactions with automation verified 
correct automation functioning more often and committed fewer errors than other 
pilots” (Mosier et al., 1998, p. 59). 

Another study that reports similar findings was conducted by Bjorklund, 
Alfredson, and Dekker (2006). They conducted an experiment to study the 
relation between the monitoring strategies of the pilots and mode awareness. 
They collected eye-tracking data, while giving participants mode transition and 
asking for status of autopilot mode. The results supported previous findings that 
pilots did not follow the standard operating procedures, but pursued their own 
strategies driven by expectations to monitor the status of autopilot. They report 
that the pilots did not capture 40% of the mode transitions. The pilots again did 
not monitor the autopilot via the SOP, and during their routine checks.. This study 
was conducted in 2006, and the findings are still similar to the studies done in the 
early 1990s. With time, one expects that the training and operational procedures 
will have addressed the common problems about automation, because it has 
been widely used for decades. However, studies indicate the existence of the 
same problems. Additionally, this is not the only recent study indicating the same 
results. 

Barter, Mumaw, and Wickens (2007) conducted a study researching the 

same issues. In their study, they mention the previous studies and point out that 

most of them used subjective measures about the pilots using the automated 

21 



systems. One of the shortcomings of previous studies using objective measures 
was the small sample size. In their study, they had a sample size of 20, and used 
a high fidelity part task trainer suitable for their study purpose. They had two 
types of events, experimentally injected and others. Other type of events 
occurred during the flow of the experiments themselves, and were also recorded 
for analysis. They also recorded the eye movements besides asking questions to 
capture the level of pilots’ understanding about automated systems. The actions 
taken by the pilots during the experiments were recorded with predetermined 
data sheets. Their findings are consistent with the previous studies. The pilots 
allocated considerably more of their attentional resources on basic flight 
parameters than the automated systems. Their automation awareness was much 
lower than their general awareness, and one of the reasons is that they failed to 
build robust mental medals about the automation (Sarter, Mumaw, & Wickens, 
2007). 

B. ALLOCATION OF ATTENTION, WORKLOAD AND TASK 

MANAGEMENT 

Doherty (2001) pointed out the most important and obvious difference 
between commercial and military aviation is that the mission in military aviation 
goes beyond basic navigation. In addition, the real missions (intercepts, target 
bombing. Air Combat Maneuvers) are as not static as level flight in commercial 
aviation; thus, the workload and management of attentional resources are more 
demanding in military aviation. 

1. Attention Theory 

Fleetwood and Byrne (2004) defined attention as a resource, and the way 
it is allocated is important because of limited resources. 

Prinzel and Risser (2004) pointed out the three sub categories of 
attention, which are more dominant in aviation and Head-up Display (HUD) 
literature, as selective, focused and divided attention modes. (Prinzel III & Risser, 
2004) 
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It is beneficial at this point to examine focused and divided attention 
modes a little bit more, to have a better insight about accidents. Prinzel and 
Risser (2004) examined these attention modes with space- and object-based 
theories. Space-based theory considers the distance between information 
sources, and suggests that the more these sources are away from each other the 
more eye movements will be required for scanning, thus resulting in a 
performance degradation due to increased scanning costs. It favors the 
information sources that are close to each other for attention to support the 
concurrent tasks requiring information from these sources. This theory, however, 
also considers the cluster effects; it will be possible to give equal attention to two 
sources if they are in “close spatial proximity,” according to Prinzel and Risser 
(2004). On the other hand, they claim object-based theories suggest that 
attention supports concurrent processing of tasks using information from sources 
that belong to the same object, defying spatial proximity. 

Besides these, Prinzel and Risser (2004) also stressed the importance of 
different domains of information on attention capture. They state that processing 
of HUD facilitates a combination of these attentional theories and information 
domains. The near and far domain each require focused attention to extract 
information when switching between the domains, while following a symbology 
that is superimposed on a target (far domain) on the HUD requires divided 
attention. 

Prinzel and Risser (2004) found that the presence of more symbology 
than is required for the current task was a factor making the attentional switching 
harder between near and far domains, due to increased clutter on the display, 
thus resulting in attentional capture or tunneling. 

They further suggested that the redundancy of the information might be 
another reason for attentional tunneling. Evidence for this is that pilots extract the 
flight information only from the HUD because it provides more sensitive 
information, and use the environmental cues just for monitoring. 
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Another possible reason for attentional tunneling suggested by Prinzel and 
Risser is perceptual load. They stated that under increased workload conditions, 
it is more likely to observe attentional tunneling because pilots cannot scan and 
filter the unrelated and unnecessary data because of the high demands from 
attentional sources. 

Prinzel III and Risser reported that “perceptual groupings” might be 
another reason for attentional tunneling. They pointed out evidence that pilots 
group informational sources according to their domains. This grouping is done 
perceptually, with near domain objects being stationary and far domain objects 
being in motion. Thus, this might be another explanation for why the far domain 
unexpected events are hard to detect. Pilots have difficulty in switching between 
these perceptual groupings. 

2. Attentional Resources Considerations in Automated Tasks 

Wickens (2000) referred to the following model to classify the data 
process stages in automation (Figure 1). 
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Figure 1. The parts of automation processing (From; Wickens, 2000, p. 2) 
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A thorough understanding of Figure 1 is necessary to understand the 
underlying reasons for attentional tunneling, and the tight relationship between 
automation and use of attentional sources. 

This figure is also significant to observe the way in which the factors 
affecting the allocation of attention implemented in the information progression 
process. It might also give a basic understanding about the attentional patterns in 
an information rich environment. Therefore, it is considered to be beneficial to 
give a brief explanation of automation, and a description of this figure. 

Wickens (2000) stated that, in the first stage, incoming raw data might be 
filtered by the system or selected to be in a certain form by the pilot. This raw 
data is represented to the pilot either explicitly or implicitly in many other forms, 
once automation determines it. All of these forms have the same effect on a 
pilot’s attention allocation process: distracting him from the other sources of 
information, they direct a pilot’s attention to a single point. 

On the other hand. Funk, Suroteguh, Wilson, and Lyall (1998) noted that, 
in automation and attention relationships: “The attentional demands of pilot- 
automation interaction may significantly interfere with performance of safety- 
critical tasks (e.g., “head down time”, distractions, etc.)” (para. 18). 

3. SA, Workload and Task Management 

Endsley (1995) referred to her previous researches while providing the 
definition of SA as: “Situation awareness is the perception of elements in the 
environment within a volume of time and space, the comprehension of their 
meaning, and the projection of their status in the near future” (p. 36). 

Doherty (2001) is one of the researchers acknowledging the additional 
required abilities for pilots in glass cockpits. One good example is the Distributed 
Aperture System in JSF. It includes several cameras all around the aircraft and 
the pilots are able to “see through” the fuselage in all directions. The DAS system 
may be prone to disorientation, which needs to be addressed in training. Doherty 
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(2001) also states that the inability to manage the ample data is the main reason 
for the “excessive workload errors” (p. 24). The pilots in JSF cockpit may be 
prone to this type of errors if they do not acquire the skills for managing the 
information in the portals in order to make their tactical decisions. 

In another study related to the same issue, Spencer (2000) reports the 
importance of the time and informational overload factors in user performance, 
and adds that a University of Toronto study concluded that time pressure was 
more important than informational overload. Although it seems that these two 
factors are different, the distinction may not be so clear in real life. In many 
situations, higher informational overloads will make the pilots’ job harder, and 
they may need more time to filter and evaluate the data, and then make proper 
decisions. If one needs to study the effects of both factors in such situations, it 
will be hard to distinguish the effect of time pressure from the informational 
overload. Either way, there remains a common outcome about the effects of 
both, and there is the need to address the related problems during training, and 
to improve the pilots’ required skills for modern glass cockpits. 

Many studies point to similar concerns, and have more or less similar 
conclusions about the changes in workload in modern glass cockpits. As a 
sensible general approach, Fiduccia et al. (2003) proposed a taxonomy of pilot 
tasks, dividing them into skills for flying the “Physical Airplane,” and those for 
flying the “Mental Airplane.” Generally, the former refers to aviating or basically 
flying the body of the aircraft, and the latter to operating all avionics, thus making 
tactical and complex mission related decisions. Fiduccia and his colleagues were 
in a team studying the human factors concerns in general aviation. Following the 
aforementioned taxonomy, they pointed out that the accidents related to modern 
systems fall into the “Mental Airplane” category, and that addressing this issue in 
training by means of hard, realistic systems scenarios should be mandatory. 
Coping with the information from all of the systems, and complicated mission 
types, the authors predict that JSF tasks will increasingly deal with flying the 
“Mental Airplane.” 
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Olson (2001) states that workload management should be addressed 
during high workload situations. In operational procedures and checklist items, 
enough time should be given for the pilots to activate and observe the automated 
functions. There might be two major aspects of automation with JSF. The first 
situation is when the autopilot commands the aircraft during high workload 
tactical mission phases. These phases are very dynamic, and the workload is 
generally very high. The pilot can easily focus only on the tactical displays and 
forget to observe the parameters managed by the autopilot at that time. Another 
issue is the dynamic changes in goals. The pilot can engage the autopilot during 
a Combat Air Patrol mission, and the goal at initiation can be maintaining a 
particular airspeed and a predefined track. But if the pilot suddenly sees an 
immediate threat, his new goal could include increasing the airspeed and 
performing high ‘g’ maneuvers. When transitioning to manual control, the pilot 
has to be aware of the current parameters and limitations of the automated 
system. 

Funk, Suroteguh, Wilson, and Lyall (1998) suggest that task management 
in advanced and highly automated cockpits may be problematic. They cite an 
incident report (#92507) from the Aviation Safety Reporting System (ASRS) 
database that indicates the significance of task management. In this incident, 
pilots of an airliner were communicating with passengers as well as making in¬ 
cockpit communications. Meanwhile, the autopilot’s target altitude changed from 
35K to 33K without any interaction. By the time the pilots recognized this, the 
altitude was 400 feet less than the cleared altitude, fortunately before any 
accident. 

lani and Wickens (2004) suggested a task hierarchy to ease the allocation 
of attention strategy of pilots under such circumstances. This hierarchy is “aviate, 
navigate, communicate and system management (ANCS)” (p. 2). 

The previously mentioned ASRS report is a good example of why task 

management is highly significant in advanced cockpit aircraft. Actual aviation is 

always the highest priority task when flying, followed in succession by the other 
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tasks. The incident cited above is another example of automation complacency. 
Over time, pilots become too dependent on automation, as well as being over 
confident about the trustworthiness of the automated systems. These kinds of 
mistakes may be critical and safety threatening, especially under high workload 
situations, in which it might take more time to recognize an automation error, if it 
could be recognized at all. 

Task management is defined by Funk, Suroteguh, Wilson, and Lyall 
(1998) as “the process by which the operators of complex systems prioritize and 
perform the multiple, concurrent tasks that compete for their attention” (para. 4). 
In this process, the operator is supposed to decide which task to attend to before 
all the other tasks, a phenomenon based on allocation of attention. 

Based upon these incident reports, in Figure 2 the authors express the 
significance of the number of incidents in advanced cockpit aircrafts. One thing 
important to note here is the decrease in the number of incidents through the 
years. Funk, Suroteguh, Wilson, and Lyall (1998) also suggested that it could be 
concluded that over time pilots became more used to automation. 
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Figure 2. Reported task prioritization incidents between given time periods 
(From: Funk, Suroteguh, Wilson, and Lyall, 1998, p. 4) 
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In Figure 3, Funk, Suroteguh, Wilson, and Lyall (1998) summarized the 
factors that could be important in task management performance. The numbers 
under the references block indicate the number of studies in the literature that 
has suggested the related line as an important factor in task management 
performance. The ones marked with on the other hand, are the suggested 
factors by the researchers, which were not previously identified in the literature. 
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Figure 3. Factors affecting task prioritization (From: Funk et al., 1998, p. 6) 


lani and Wickens (2004) found that task switching time was directly 
related to salience of the cue. They suggested that: “people tend to be more 
proactive in task management when workload is low and more reactive when 
workload becomes high” (p. 2). 
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lani and Wickens (2004) stated that even if the task prioritization order 
should follow ANCS hierarchy in an effort to allocate the attention ideally, this is 
not always the pattern that performers follow. They noted there were accidents 
because of devoting attention to one task while a higher priority task was 
neglected, lani and Wickens also referred to an accident as an example of this 
kind of neglect, when the pilots of an airliner were distracted by a landing gear 
indicator failure. Pilots devoted their attention to this malfunction, resulting in the 
neglect of aircraft control, and failure to recognize the descent of aircraft. Thus, 
the airliner crashed with the death of all on board. 

Freed (2000) focused on the task prioritization computation of agents, and 
suggested a heuristic prioritization, when all other factors were equal, with the 
following order: 

First is urgency, which is the time period until the deadline of the task, 
suggesting completion of the nearer deadline first. Second is importance, which 
depicts the consequences of missing the deadline of a task, suggesting 
completion of the most important deadline first. 

The third factor—duration—is important for two reasons. First, the task 
duration might affect the completion of other tasks before their deadline, and 
second, depending on the duration of tasks and given deadlines, the task 
prioritization order would change, therefore suggesting completion of the briefer 
task first. 

The last place in this heuristic goes to interruption cost, which depicts the 
attention switching costs from one task to another, suggesting completing an 
ongoing activity rather than switching to another (Freed, 2000). 

lani and Wickens (2004) gave the factors affecting task prioritization in the 
following order: task complexity, attentional tunneling, task importance, and 
physical salience. They state that switching attention to new tasks depends on 
the urgency of the ongoing task, and the attentional resources it allocates. While 
there is an attentionally demanding ongoing task, there will be fewer attentional 
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resources to attend to the other tasks. At this point, attentional tunneling comes 
on the table, because it is not always the case that the operators use their 
attentional resources on the more complex tasks, instead attending to less 
complex but more urgent ones. Therefore, lani and Wickens concluded that the 
urgency of the ongoing task, not its complexity, is the most significant factor 
when it comes to switching attention between concurrent tasks. Independent 
from urgency and complexity there is task importance. Operators should 
compare the importance of tasks while allocating their attentional resources, 
suggest the authors. However, they suggest the physical salience of the task 
here to explain the fact that tasks are not always initiated due to their importance. 
They pointed out evidence that memorial or less salient reminders are less likely 
to initiate an attentional switch than the salient reminders. They also pointed to 
evidence that auditory salience is more effective in switching attention to new 
tasks than is visual salience, and the switching time of higher priority interrupting 
tasks was much faster than the lower priority ones. Their findings support this 
evidence, and expand it by stating that auditory modality does not deprive high 
priority flight tasks from attentional resources because it works parallel to the 
visual flight parameters. 

On the other hand, Damos and Tabachnick (2001) found that task 
duration and structure had significance on interruption of the ongoing tasks. It 
was seen that ongoing tasks that cannot be divided into sub-tasks and are 
relatively shorter in duration were not likely to be interrupted. They also stated 
that something which occurs frequently would indicate less time and safety 
critical information, thus would be most likely to be assessed as too low priority a 
task to interrupt the ongoing one. 

Prospective memory was explained by Berg (2002) as the memory where 
the intentions to take certain acts are kept for the future. It is expressed that 
acting upon remembering is significant, rather than just remembering those 
intentions. 
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In a recent study, Dismukes (2006) analyzed aviation accidents where 
crew error was the major component of the reason. He found out that neglecting 
the regular operational sequences was a significant factor of crew related errors. 
To find out the underlying reasons for such errors from experienced pilots who 
have done the same operational procedures countless times, he ran a study and 
argued that prospective memory along with multi-task management constituted a 
reason for human error. 

Dismukes (2006) characterized prospective memory by the following 
traits: the intention to take an action in the future when conditions permit, time 
between intention and its performance (which is allocated with other tasks), and 
the absence of an indication to remind it to the performer explicitly. He stated that 
the crucial issue is to retrieve these intentions from the memory when the 
circumstances permit, but not to keep them in the memory. He suggested that 
prospective memory plays a significant role in the following task situations in the 
cockpit: episodic tasks, habitual tasks, atypical actions substituted for habitual 
actions, interrupted tasks, and interleaving tasks and monitoring. He stated that, 
unlike habitual tasks, episodic tasks have to be remembered to be performed. 
Dismukes added that in the information rich environment of cockpits, the pilot’s 
attention might be allocated to some other cues rather than the ones that would 
help to retrieve episodic tasks. Habitual tasks, on the other hand, hold a risk of 
omitting some important steps, especially when the sequence of operations is 
changed for some reason. Another phenomenon related to habitual tasks is 
when the pilots need to substitute their habitual procedures under some 
particular circumstances. Dismukes stated that pilots might attend to their 
habitual procedures rather than the substituted one. He pointed out a common 
error where pilots do not return from their interrupted tasks after accomplishing 
the interrupting task. Findings of his study showed that the reason for this 
behavior was not having explicit cues in the perceptually rich environment of the 
cockpit. Another reason was the oncoming task demands right after the end of 
the interrupting task, which prevented the participants from comprehending the 
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situation and retrieving their prior intention from the memory to return to the 
interrupted task. Another task demand that might cause prospective memory 
retrieval issues is when the pilot needs to shift attention to monitor another task, 
while performing an ongoing task. Dismukes (2006) stated that even if their 
consequences might be significant, monitoring low probability events might be 
difficult because a human has a tendency to allocate his attention to the sources 
where he can get more information. 

Hancock, Williams and Manning (1995) stated that human performance is 
not a linear phenomenon that works directly proportional to the given task 
demand. The authors mentioned that automation has changed the pilot’s role 
from operator to system manager. They referred to literature about two contrary 
opinions about the task demand, workload and human performance relationship. 
The first opinion claimed that workload is directly related to the task demand 
characteristics under circumstances where attention is directed to the source for 
prolonged periods. On the other hand, it was claimed that workload and task 
demand are separate phenomena, and under some circumstances, workload 
and human performance are directly proportional to another, where human 
performance increase is observed as workload increased. 

Considering this background, Hancock and his colleagues made a study 
to point out the relationship of task demand, workload and performance. Their 
findings indicated that the perceived workload of a performer is related to the 
level of task demand he was subject to until that time (e.g., pilots who 
transitioned from a relatively higher task-intensive platform perceived the 
workload as low, while pilots transitioning from low task-intensive cockpits 
perceived the given workload as high). Therefore, the authors suggested that to 
better assess the current workload level, task/mission history has to be taken into 
account. This perception of relative workload leads to the conclusion that pilots 
coming from high to low task-intensive environments should perform better and 
vice versa. The initial reflex is to associate high workload with poor performance, 
while linking low workload and good performance. However, the authors 
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observed that transitioning from a higher task-intensive cockpit to a lower task¬ 
intensive one did not produce higher human performance, but lower. The authors 
suggested that workload analyses provide a window of efficiency which may 
enable proactive behaviors in cockpits by potentially providing information before 
the fact happens. They suspected that the reason for uncertainty in correlation is 
nonlinear human characteristic, because of some observations where 
performance increased while the tasks became harder. They commented on the 
promising findings where performance and workload appear to be associated in 
monitoring tasks, which constitute an increasing percentage of the tasks to be 
done in each new cockpit design. 

In a study to examine mental workload and situational awareness, 
Alexander and Nygren (2000) stated that mental workload is important in 
assessing systems, but that when assessing the quality of information that the 
operator is using, situational awareness should be also considered. In their 
experiment, Alexander and Nygren compared two different cockpit interfaces: a 
conventional one, and a virtually augmented cockpit with advanced displays. 
They found that mental workload was lower in the virtually augmented cockpit 
than in the conventional one. 

Alexander and Nygren also stated that having high situational awareness 
enables pilots to function more effectively and with more time awareness. They 
mentioned that even if situational awareness and mental workload are inter¬ 
related, they are considered to be independent elements, referring to the findings 
where mental workload ratings increased with the increase in task demands 
while situational awareness ratings did not change. However, their findings 
indicated a relationship between mental workload and situational awareness. 
They observed that changes in the experimental set up which caused mental 
workload increase resulted in a decrease in situational awareness. 

Wilson (2002) examined pilot workload using measures such as heart 

rate, eye blinking rate and the electroencephalographs (EEG) of pilots. He stated 

that increasing heart rate would indicate increasing mental workload, and that 
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decreasing blink rate would indicate increasing visual demands. He noted that 
these psycho-physiological measures changed more than the other phases of 
flight at takeoff and landing phases. He also showed that when pilots flew similar 
maneuvers to the ones they did before, their workload ratings were lower than 
when they conducted less familiar maneuvers. 

4. Auditory and Visual Resource Considerations 

In a current study, Lee, Lee and Boyle (2007) examined the effects of 
voice interactions of drivers to assess the effect on their attentional guidance. 
They predicted that voice interactions with in-vehicle systems, while enabling 
them to keep their visual focus on the road, would cause additional cognitive 
load. Therefore, this load would cause delays in their responses to regular events 
such as braking and showing reaction to traffic lights, when responding verbally 
to auditory messages rather than just listening to them. In their experiment, the 
main task of their participants was to follow a frequently braking lead car at a 
certain distance. Concurrently, the participants were supposed to follow and 
remember signs, pedestrians and various similar targets in the scene, and listen 
to an auditory message, then respond to questions about this message. This 
experimental design composed a complex dynamic situation containing many 
tasks required to be done more or less concurrently. 

The findings of the study supported their predictions. Lee and her 
colleagues found that drivers showed slower reactions under dual-tasking 
conditions, where both tasks required a response from driver. This finding, 
however, was mostly the result of attentional distraction, because the drivers had 
to respond to asked questions, but not because of the additional information 
given by the auditory message. 

Similar to this finding, Damos and Tabachnick (2001) reported that when 
the ongoing task in the cockpit was auditory, pilots reacted slower to the 
interrupting tasks. 
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The findings of lani and Wickens (2004) established the basis for these 
suggestions, where they stated that the auditory modality, in fact, is a factor that 
supports the parallel processing of visual control sources. 

According to Wickens and Ververs (1998) the main reason for putting 
HUDs in cockpits is to present necessary information to the pilot on one source, 
so that he can save his attentional sources for higher priority tasks rather than 
using them to re-accommodate while switching from one display to another to 
extract the necessary information. Thus, it is critical to understand the way 
attention is modulated. 

Wickens and Ververs (1998) pointed out evidence that a HUD creates 
attentional narrowing, especially under high workload situations, thus avoiding 
the assessment of presented data on the HUD by the pilot. They referred to a 
military report to indicate the possible threats of attentional narrowing caused by 
HUDs. In this incident, the pilot failed to notice the barrier on the runway due to 
overloaded symbology on the HUD, in addition to the excessive brightness, 
resulting in an accident. 

Wickens et al. (2004) suggested that one significant issue is “to evaluate 
the general tradeoff between the scanning costs of a separated display, and the 
clutter costs of a more integrated display” (p. 5). While the clutter costs for target 
detection have been observed before, they found evidence that symbology 
clutter even degrades the detection rate of expected targets. However, in some 
cases, the benefits of some sources of clutter were great enough to dominate 
their costs, because they reduced the workload. 

The data in Wickens and colleagues’ study revealed that pilots use the 
synthetic display to extract the attitude information even if they can see the 
horizon in IMG. This attentional preference establishes a significant threat to 
pilots, especially when there is information outside which is not presented on the 
synthetic display. This finding supports the automation complacency risk 
suggested by Wickens (2000). 
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Wickens and Ververs (1998) referred to basic attention and aviation 
literature to point out the findings that show the negative effect on attention of 
having unnecessary symbology and visual data. They state that it is highly 
significant to determine which data is required within the context of any given 
flight phase, following which the rest should be cleared accordingly. Their results 
indicated that high clutter disrupts the target detection performance both in near 
and far domains. 

The presence of more symbology than is required for the current task was 
pointed out to be a factor making the attentional switching harder between near 
and far domains, due to increased clutter on the display, thus resulting in 
attentional capture or tunneling. 

Yeh, Wickens and Seagull (1999) pointed out that the scanning cost of 
head-up display clutter is less than the scanning costs of head-down instruments. 
However, they also stated that the cost of scanning of head-up displays 
increases significantly with the increasing symbology on these displays intended 
to present more information to the pilot. They claimed that previous research has 
proved that pilots are more vulnerable to miss the information in the real world 
and make errors with a cluttered display in their field of view because of impaired 
vision due to increased symbology. 

Wickens and Ververs (1998) analyzed the scanning vs. clutter costs, and 
their influence on near and far domain target detection. They reported that the 
reduction in the costs of attention switching between the head-down displays and 
the scanning cost outweigh the clutter cost of HUD in detecting environmental 
targets. HUD decreased the detection time of far domain targets significantly. On 
the other hand, clutter adversely affected the near domain target detection 
performance when it came to recognizing the change on the HUD symbology, as 
well as far domain. 

Prinzel and Risser (2004) commented that being attentionally captured 
because of the salience of the near domain cues, pilots experience difficulty in 
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switching their attention between domains; thus, they cannot process two 
sources of information at the same time. They pointed out pilot reports, where 
several of them admitted that they sometimes find themselves being so fixated 
on near domain cues that they are totally unaware of anything else. 

Wickens and Ververs (1998) found that pilots prefer to use the 
environmental sources to extract attitude information, which would explain why 
they do not keep their eyes down on the flight instruments as much as they 
might. These results favor the use of HUD over head-down instruments, of 
course. 

Yeh et al. (1999) found out that cuing symbology on HMD was useful for 
expected targets and helped subjects to point out them out, while distracting 
them from unexpected ones. They pointed to evidence from an air-to-ground 
mission experiment indicating that cuing symbology resulted in erroneous 
decisions, making pilots target non-target locations. 

This result is compatible with the habits of pilots using HUD. In an air-to- 
ground attack, pilots continuously track the approximate target location while 
pulling the aircraft to that direction to put the target in the field of view of HUD. 
When the HUD field of view covers the area, the cuing symbology is seen 
overlaying on the target. This order enables pilots to avoid computer generated 
error, as well as human error, because it gives opportunity to double check and 
assess whether the correct target has been acquired. Therefore, it is also fair to 
claim that pilots using HMD are more susceptible to targeting errors unless their 
systems work perfectly, because the continuous symbology in the field of view is 
going to discourage them from focusing their attention on the far domain. 

Yeh et al. (1999) stated that when there is a contradiction between the 
information acquired from the real world and the information from computer 
generated symbology (automation); the pilots’ decisions were to trust in the 
automation-based information. Their results support this claim by stating that the 
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presence of cuing information might draw attention to a certain area, thus 
withdrawing attention from surrounding areas. 

Yeh and colleagues also suggested that cuing symbology on HMD’s could 
increase attentional tunneling, after their experiment about using an HMD and 
computer-generated fault imagery for aircraft inspection. They also stated that 
“the presence of cuing may result in an inappropriate allocation of attention: an 
overreliance on an automation-based cue” (p. 539-540). Further, they suggested 
that as the amount of information presented in the field of view of the pilot 
increases, the target detection rate decreases because of the allocation of the 
pilot’s attention to the near domain. 

Johnson, Wiegmann and Wickens (2005) found that as the attention 
allocated to a certain area of interest increases, the task performance of the 
pilots, which require other areas of interest, decreased proportionally because 
the pilots could not allocate the necessary amount of attention to these sources. 

Johnson, Wiegmann and Wickens also stated that the performance of 
pilots on certain tasks that are based on visual scanning is also negatively 
affected by hi-tech cockpit displays, as is the visual scanning itself. They 
suggested that if pilots’ visual scanning is interrupted by the in-cockpit displays, 
thus drawing attention into the cockpit and away from the world outside, their 
assessment on weather conditions will become weaker over time. 

Wickens (2002) indicated that salient display cues are beneficial to remind 
pilots of the tasks to be done, thus mentioning the importance of allocation of 
attention to better task management. Connected to this statement, Wickens 
criticized the task management strategies that direct the pilot’s attention to a 
certain direction to inform him about the flight data. He stressed the importance 
of training in task management by defining what makes a pilot a better task 
manager, and stated that the key to becoming a better pilot is the ability to share 
the attentional sources between the outside world and the informative cockpit 
displays. 
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Bohnen and De Reus (2004) stated that one of the most significant 
indicators of mental workload is the visual allocation of attention, and used it as a 
manipulator to find out the effect of number of displays on pilot workload. Their 
findings indicated that as the number of displays associated with concurrent 
tasks increased, pilots could not maintain the same flight performance because 
of limited attentional sources. However, they also found that increasing the 
number of displays did not bring additional mental load, thus pointing out the 
visual allocation of attention as the reason for performance decrease. On the 
other hand, they observed that pilots developed new strategies to manage 
attentional sources and maximize flight performance while doing the other 
concurrent tasks. Thus, the authors expressed the importance of considering 
pilots’ attentional management strategies next to visual allocation of attention 
while predicting the workload. 

Horrey, Wickens and Consalus (2006) stated that as more technologies 
are implemented in cars, a safety concern arises because these new 
implementations are going to compete for the limited visual and attentional 
resources of drivers, thus reducing the resources allocated for the primary task, 
driving. They claimed that the eye is kind of a “single-server queue” and visual 
scanning is the server of this queue. 

Horrey et al. (2006) expressed the importance of expectance and 
information bandwidth by stating that performers become more likely to monitor 
the displays/information sources where they find relevant information more 
frequently than others. They showed evidence of this phenomenon in that the 
experienced drivers were observed to have more extensive visual scanning while 
novice drivers became fixated on cars around them during a task where they had 
to track their lane in a demanding road environment. This observation, the 
authors suggested, indicates that experienced drivers have a better idea of 
where to expect and extract the relevant information than do novice ones. 
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Horrey and his colleagues suggested that a scanning pattern should 
optimize the cost-benefit effectiveness, where it increases the benefits of 
scanning while decreasing the costs of missing information. 

In an experiment, Horrey and his colleagues (2006) used the SEEV 
(Salience/Effort/Expectancy/Valuable) model where they intended to point out the 
influence of in-vehicle tasks on driving performance and visual scanning patterns. 
They explained the parameters of this model as follows: salience is the 
obviousness of information on an information source, effort is the visual angle 
difference between informational sources, expectancy is a combination of 
bandwidth and value of the information on a display which indicates the task 
relevance and importance of information for the given task, and value is the 
importance of that information compared to the others. The authors pointed out a 
weakness of the SEEV model by stating it is built on the “momentary allocation of 
focal (foveal) vision” while there are various tasks where necessary information 
can be extracted through ambient vision (p. 68). This evidence indicates that 
performers do not necessarily need to fixate their attention on focal vision for all 
the information sources. Focal vision is said to be linked directly to eye 
movements, and used for visual search and tasks such as reading, which require 
“high visual acuity,” while ambient vision is mostly used for spatial orientation, 
and postural control in locomotion. 

Prinzel and RIsser (2004) stated that a large part of the world in humans’ 
vision is continuously monitored by ambient vision, and only a smaller part by 
foveal vision. They stated that these two vision systems work in a parallel 
fashion, without competing for the same attentional source. 

Horrey et al. (2006) showed evidence that experienced drivers can use 
their ambient vision more effectively than novice ones for lane tracking, even 
without moving their eyes to direct their attention on the road. 

The findings of Horrey and his colleagues showed an increase in 
performance proportional to the increase in allocation of focal visual sources that 
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was related to the increase in value of that particular task. Therefore, it has been 
seen that as the priority of that task increases, more attention is directed to that 
task, which results in better performance. Their findings also indicated that as the 
complexity of in-vehicle systems and their bandwidth increases, the lane-keeping 
performance decreases due to switching attention from the road to in-vehicle 
displays. Drivers’ response time to safety critical situations, which is dictated by 
the time for the eyes to move from down inside the vehicle to the road, also 
increased, pointing out the role of focal vision. However, the correlation between 
focal vision and hazard reaction times is not that strong, suggesting that ambient 
vision is also used for hazard detection and reaction, though still requiring focal 
vision to correctly identify and react. This study examined only one-task 
conditions. No multi-task performance was examined, so the increase in task 
performance when the allocation of attention to one source might not prove to be 
beneficial under high workload conditions. However, the findings are significant 
to support the importance of task prioritization in allocating the attentional 
resources to the required task. 

Horrey and his colleagues’ (2006) results showed that increasing the 
frequency or complexity of in-vehicle systems influenced the driving performance 
more than the manipulation of outside factors, indicating the importance of 
allocation of focal vision attention. 

5. Decision Making Considerations 

Decision-making often becomes very demanding in the cockpits. In order 
to mitigate the problems, it is crucial to understand the decision-making process, 
and better decision-making practices, via the literature. 

Doherty (2001) mentioned Klein’s “recognition-primed decision making 
model (RPD)” (p. 15). The model says that experienced pilots relate their current 
situations to their stored situations from their experiences and make correct 
decisions easily. Considering the capabilities of JSF, this kind of decision-making 
ability will be crucial. In high-risk, limited time situations and with lots of available 
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information via the displays, it will be very important to make the correct 
decisions quickly. As Doherty mentions, the required experience for RPD can be 
built in various ways such as real or training missions and training systems. 

Similarly, Spencer (2000) discussed that in many situations, the experts 
make quick decisions depending on their experience. Over time and training, 
they build a repository for many situations, and if a critical situation requires a 
very quick reaction, they retrieve the closest match and decide their next move. 

The multirole capability, increased number of complicated sensors, and 
sophisticated autopilot are the main features of JSF that create differences from 
previous fighters. If one also considers the missions that JSF will perform in the 
future, it becomes obvious that the pilots will be under high workloads in limited 
time frames while performing their tasks. Thus, properly trained automated 
responses, like RPD, will become more crucial. 

A typical mission scenario example will be very explanatory in terms of 
mission nature; during composite air operations (COMAO), where multiple Air-to- 
Air, Air-to-Ground, friendly and hostile assets are in the battle area, it will be 
highly likely that a JSF pilot will be looking at a complicated picture with a lot of 
information in the cockpit. Considering the time constraints, and other stressors, 
it is easy to conclude that the pilot’s ability to look at the displays, filter the data, 
and evaluate it for a quick and correct decision will be very challenging. And from 
the beginning of their exposure to JSF, the pilots will have access to all of these 
sources of information. Considering the number and the complexity of the 
systems, a considerable amount of training will be needed to achieve the ability 
of accomplishing the aforementioned tasks. 

6. Models to Predict Allocation of Attention 

Even if the prediction of attention allocation is beyond the scope of the 
current study, the authors of this thesis believe that briefly mentioning some 
models can help to understand how the process of human attention is captured. 
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Fleetwood and Byrne (2004) analyzed three models of visual attention 
allocation to point out the attentional patterns and where the performer is going to 
direct his attention next. They focused on the following factors to determine to 
what extent they influence the attentional patterns of the performers: the 
frequency of information generation on the monitored display, the probability of 
critical information coming up while monitoring another display, the cost of 
missing/detecting critical information, and the cost of monitoring. 

The first model they analyzed was the Senders model. This model 
concentrates on the frequency of information generation (this factor also is called 
the bandwidth of related instrument/display) and the alarm frequency of this 
instrument. However, Fleetwood and Byrne expected this model to predict less 
accurately when under complex task conditions because only the bandwidth can 
be manipulated. This expectation might indicate that this model is less likely to be 
used in aviation, where the complexity increases due to higher workload. 

The second model Fleetwood and Byrne examined is the SEEV model. 
The name of the model stands for Salience, Expectancy, Expectancy and Value, 
which constitute the factors that this model considers to influence the allocation 
of attention. Authors expect a high and relatively accurate prediction rate from 
this model, unless its results are extrapolated for more specific ones. Fleetwood 
and Byrne stated that the information extracted from a source, the effort to 
monitor that source and the effort to switch attention from one information source 
to another are the factors considered by the Information Foraging model. They 
said that SEEV and Information Foraging models predict that the sampling 
frequency of an information source is relevant to the value of the information 
extracted from that source. 

7. FAA Research about Flight Training 

The main consequences of modern aircraft are the enhanced capabilities, 
increased number of sensors, and automation. A very important and useful set of 
research related to the human factors concerns in modern cockpits is the 
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research conducted by the FAA about general aviation. The research began 
approximately in 2003 with a final report analyzing the mishaps of modern 
general aviation aircraft, and continued to evaluate almost to the present day. 
Another important aspect of the research is the solutions proposed about the 
training. 

The first related study is the final report by Fiduccia et al. (2003). Their 
study is based on 11 accidents, all of which had issues related to 
“Technologically Advanced Aircraft” (TAA). Their definition of TAA is: 

a. IFR-certified GPS navigation equipment (navigator) with moving 

map; or 

b. A multi-function display (MFD) with weather, traffic or terrain 

graphics; and 

c. An integrated autopilot, (p. 9) 

This definition has similar parts to modern fighter jets, in terms of the 
MFDs, GPS and autopilot. It is legitimate to see their study as a case study on 
the accidents by Subject Matter Experts. And as an important conclusion, 
Fiduccia and his colleagues claim that the traditional flight training did not 
address the use of relatively modern technological systems, and that a study on 
how to address all required issues of the new technology needed to be 
conducted. This study established the basis for the following related studies that 
sought solutions to this common training concern. 

French, Blickensderfer, Ayers and Connolly (2005) conducted one of the 

follow-on studies in order to study the differences between the “maneuvers 

based training (MBT)” and the “scenario-based training (SBT)” (p. 3). The MBT is 

generally the approach of the traditional instrument training used so far, and the 

SBT was the proposed solution by the TAA research team after conducting 

research on the accidents in TAAs. They explain that SBT is a student centered 

training approach in which the students declare their problematic areas and build 

appropriate and realistic training scenarios to improve them. After this process, it 

is the instructor’s responsibility to come up with appropriate “performance 
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measures.” Then, the instructor decides the success of the trainee using those 
performance measures and gives feedback. The main point of SBT is that “...the 
scenario is the curriculum” (p. 6). French and colleagues also explain the 
traditional training approach as based on MBT. The repetition of the required 
maneuvers makes the trainees proficient. The maneuvers are not implemented 
to scenarios, and the student only follows the syllabus, but this does not 
contribute to the flow of his training with deciding what to practice in which 
realistic scenario. 

French and colleagues conducted their study with 27 participants who 
were not experienced in TAA. Those participants ran through a pretest, received 
8 hours of training with either MBT or SBT, and then completed another test 
thereafter. They divided the phases of flight into eight categories and graded 
them separately with a “blind rater.” Additionally, they also used subjective 
surveys in order to learn personal opinions and get deeper insight. Their results 
revealed that the trainees with SBT received higher grades than MBT trainees in 
five out of eight categories with a statistical significance, and in the remaining 
categories their grades were equal. The authors also add: “Further, the SBT 
group demonstrated a tendency to report reduced workload and an improvement 
in self efficacy and situational awareness compared to MBT” (French et al., 2005, 
p. 3). 

Robertson, Petros, Schumacher, McHorse and Ulrich conducted one of 
the FAA sponsored studies in 2006. The purpose of the study was to measure 
the differences between the “problem-based learning” studied by the FITS and 
the traditional training approaches. In PBL, the scenarios should be prepared 
such that they force the trainees to face more challenging and harder cognitive 
processes, and to evaluate and compare more than one alternative during the 
decision making process. They reported that FAA has accepted the 
shortcomings of the current training approaches in “atypical” conditions. 
Regardless of whether a situation was normal or abnormal, it was designated as 
typical when the standard response to that situation was covered in the training. 
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They defined the “atypical” situations or hard conditions as those with many 
alternative solutions that were complex, and which were not covered during 
standard training (Robertson, Petros, Schumacher, McHorse, & Ulrich, 2006, p. 
2 ). 

To teach the required skills to pilots for giving them the ability to make the 
proper decisions during the atypical conditions, it is necessary to address the 
complex or advanced decision making skills during the training. And they 
reported that the current training systems do not address this issue; that was one 
of the objectives of their study to evaluate “Higher Order Thinking Skills” in their 
training system (Robertson et al., 2006, p. 3). 

Robertson and colleagues conducted an experiment using a simulation 
program in a personal computer. The three groups were used for a FITS 
preferred PBL approach, or the traditional training approach and self study group. 
Each group was given a test before they began their training with a TAA aircraft 
simulation, went through their training and at the end took a posttest of their new 
TAA aircraft. Their participants were college students who had private pilot 
license and were certified to fly in Instrument Flight Conditions. The traditional 
and self-study groups were given traditional training during the ground lessons, 
briefings and flight lessons. They were asked multiple choice or straightforward 
questions requiring direct answers. The PBL group was given questions via the 
scenarios that required the students to do further research with their documents. 
Also, the answers to these scenario-based questions were not as straightforward 
as the other groups’ questions; rather, they necessitated more intensive mental 
efforts (Robertson et al., 2006). 

Both subjective and objective measurements were used. The simulation 

game had the feature of data collection and saving. That feature was used to get 

the objective measurements about the parameters of interest such as altitude, 

airspeed, etc. SA was measured by the questions asked by the researchers, 

while they stopped the simulation. “Aeronautical Decision Making” was measured 

subjectively by the observing researchers, and objectively by examinations about 
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the HOTS. They reported that the participants with PBL performed significantly 
better under complex conditions, and added “The findings also reflected 
improvements in the indicators of aeronautical decision-making (pilot judgment) 
and a reduction in the number of mistakes made by the pilot” (Robertson et al., 

2006, p. 60). 

Dornan, Beckman, Gossett and Craig (2007) are the last researchers of 
the FAA research. One of the recommendations of their report was adding 
“consequences” to scenario-based trainings. They claimed that without any 
consequence, the scenario-based training would not be fully useful, because to 
come up with a solution will require less effort. But when adding serious 
consequences to the scenario, the trainees will be forced to process more 
information, and use more cognitive resources; thus, the end result of the training 
will be more beneficial. The example they had for such an input was injecting a 
“...transplant organ to the destination airport” in a possible divert decision 
scenario. Apparently after going through that kind of hard, challenging decision, 
the trainees will gain more than without any consequence injected training (p. 3). 

Another method of exploring the proper decision-making skills is looking at 
the experts. Many studies have been conducted for that purpose, and point out 
similar phenomenon. Spencer (2000) discussed this common finding that in 
many situations, the experts make quick decisions depending on their 
experience. Over time and training, they build a repository for many situations, 
and if a critical situation requires a very quick reaction, they retrieve the closest 
match and decide their next move. 

The cockpit automation, load on visual resource, informational overload, 
cognitive resources, attentional allocation, training considerations and many 
other topics covered in this chapter provide a background on all of the required 
aspects of human factors concerns in modern glass cockpit aircraft. The 
following chapters benefit from the relevant studies mentioned in this chapter, 
both in problem identification and training recommendation purposes. 
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C. THE IMPORTANT FEATURES OF JSF RELEVANT TO HUMAN 

FACTORS 

One of the important objectives of this thesis lies in understanding the 
features and capabilities of JSF. The authors believe that JSF has many new 
capabilities and systems leading to new required skills and human factors 
concerns. In this chapter, these topics as well as the differences among fighter 
cockpits are provided. First, the features and major systems of JSF are 
introduced from unclassified documents, and then possible human factors 
concerns are predicted from the literature and the operational experience of the 
authors. 

1. General Features and Systems of JSF 

JSF is intended to be a "... multi-branch, multinational, supersonic fighter,” 
which is planned to replace most of the current generations of aircraft in the 
contributing countries’ air force inventories. With its advanced features and three 
variations for different platforms, it offers multi-role capabilities. Fusing the data 
from many sensors onboard the aircraft is claimed to be the major strength of its 
design (Jensen, 2005, para. 2). Even if the flight performance, maneuverability 
and G-performance are not necessarily superior to the previous generations, the 
systems presented with this aircraft offer new capabilities and operational 
concepts. Next to the improvements on already existing systems, some highly 
intuitive systems are introduced to the aviation community for the first time, such 
as the electro-optical distributed aperture system (DAS). Even if there are many 
other points to consider assessing JSF, its capabilities, and pilot vehicle 
interface, the authors think the following systems are more likely to affect mission 
effectiveness. 

The most striking difference and innovation of the JSF cockpit is the 
8”x20” liquid crystal display screen right in the middle of the forward panel 
(Figure 4). Almost all of the information from the sensors and systems are 
presented to the pilot via this display, which can be modified by touch screen 
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buttons to reach and modify any desired system data. This display will be the 
major PVI system between the pilot and aircraft. 



Figure 4. JSF cockpit with LCD display suite (From: briefing received from PVI 

Team at Lockheed Martin, Fort Worth, TX) 


The forest of toggle switches in previous fighter cockpits has been 
wiped clean from the F-35’s interior landscape, with most of their 
functions moved to the touch screen. A few switches still sprout 
here and there, but the overall cockpit ambience is one of 
simplicity and calm, almost to the point of aeronautical feng shui. 

(Kent, 2006, para. 3) 

These features give the impression that it will be very intuitive and easy to 
operate the systems of JSF. 

After providing a general picture about the capabilities and cockpit design 
of the JSF above, it will be necessary to focus on the individual systems. The 
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following systems and their features do not cover all of the systems and 
capabilities of JSF, but are thought to be the important ones relevant to the 
scope of this thesis (related to human factors concerns). Limited information can 
be found online and in some published resources. But because JSF is still under 
development, there is much restricted information due to security concerns, and 
the following information is cited directly from unclassified sources online. For 
further information, the readers can follow the references. 

2. Radar 

The AN/APG-81 fire control radar, developed by Northrop Grumman, has 
the following general features: 

• Terrain mapping with high resolution, able to cover an area three to four 
times wider than existing radars. This feature enables pilots to have better 
assessment of the area of interest by giving high resolution and wider field 
of view (Jensen, 2005). This feature might also introduce attentional 
tunneling issues along with itself because pilots might fixate their attention 
on this display to search and acquire the target. 

• Provides both air-to-air and air-to-ground target information at the same 
time. This is a significant capability to provide the pilot a better situational 
awareness in a multi-threat environment, where he can assess all the 
given parameters and make quicker and more accurate decisions 
(Jensen, 2005). This capability is not present in current generation 
fighters. Flowever, it may present a challenge to informational 
management, where the pilot should be able to filter the unnecessary data 
and focus on the required ones. Pilots, apparently, are required to manage 
their mental resources more effectively in order to cope with ample data 
and make better decisions. 

• It can be slewed to any other sensor’s field of view, whether the other 
sensor is onboard or off board (Jensen, 2005). This feature amplifies the 
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situational awareness and the coordination among friendly assets; 
however, given information does not allow any further assessment in 
terms of human factors concerns. 

3. Electro-optical Targeting System (EOTS) 

Along with the AN/APG-81 radar, the EOTS is one of the major sensors of 

JSF. 

• It consists of non-active sensors operating both during night and day; 
enemies are not able to intercept any signals because it doesn’t radiate 
any signal. 

• It provides very detailed IR images to the pilots, and is said to be an 
additional way of acquiring and analyzing targets in addition to the radar. 
The pilots will be able to acquire the targets beyond visual ranges, and 
sync the radar with EOTS, in order to examine the targets in more detail. 

• For Air-to-Air purposes, it has an Infrared Search and Tracking System 
(IRSTS), and for Air-to-Ground purposes a Forward Looking Infrared 
Radar (FLIR). Any off-board system can target laser to target, and those 
lasers can be locked by a tracker in JSF (Jensen, 2005). 

Apparently, the pilots will focus on both radar and EOTS in many task 
situations. Focus and attentional distribution as well as having robust mental 
models on both systems will be crucial with such advanced systems. 

4. Distributed Aperture System (DAS) 

The electro-optical distributed aperture system (DAS) has the following 
features: 

• Consists of six IR cameras. 

• Provides vision through the body of the aircraft. 

• Works in collaboration with the helmet mounted display (HMD) to relay a 

continuous 360 degrees of passive environmental information to pilot. 
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• Gives the pilot “missile approach warning, countermeasures deployment, 
passive air-to-air radar, off-axis targeting for air-to-air missiles, and wide 
field-of-view day/night pilot vision” (Jensen, 2005, para. 20) 

• The integrated data from all cameras can be superimposed over a tactical 
display which can be reached via data link from ground units or another 
flying asset (Jensen, 2005). 

The DAS intends to increase the situational awareness of the pilot while 
not overloading him with excess information. The system itself, along with its 
ability to work in collaboration with data linked tactical pictures, is intended to 
ease the pilot’s workload of adapting himself to the tactical arena, 
comprehending, deciding and taking action (Jensen, 2005). 

However, being such a new system, these authors think that it will take 
some time and operational experience to be thoroughly assessed by means of 
flight safety in certain conditions. Previous studies done on visual attention 
allocation are helpful to consider the possible attentional tunneling issues 
associated with this system and its presentation on HMD. However, future 
research should be encouraged by air forces operating this aircraft, not only 
depending on simulator experience but also pilot feedback from real operational 
missions in various weather and mission scenarios. A question that arises is pilot 
disorientation. What kinds of effects will the DAS cause to pilots after “looking 
through” the fuselage and returning back to normal vision in bad weather 
conditions? These types of concerns should be considered during training to 
cope with any possible problems. Another issue is the balance between the 
capabilities of the aircraft and the pilots. Apparently, the aircraft has the potential 
to dominate the pilot with ample data and many other capabilities. But how to 
raise pilots’ capabilities to a point that they can fully benefit from the data safely 
and efficiently without fixating or becoming disoriented seems to be a challenge 
during training. 
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5 . 


Communication/Navigation/Identification (CNI) and Other 
Capabilities 


The CNI system will be approximately similar to the most modern current 
generations, but the officials report that it will be more tailored to fit in “network 
centric warfare” (Jensen, 2005). The system is also reported to provide such 
functions as beyond-visual-range identification friend or foe, secure voice 
communications, caution and warning, intercom, and intraflight information 
sharing among multiple aircraft via high-speed broadband data link” (Jensen, 
2005, para. 29). JSF will also have other current data link capabilities to share 
information (Jensen, 2005). The aforementioned capabilities will take JSF one 
step further in terms of information dominance, and common tactical picture; 
thus, ability to filter and evaluate the data, and make proper decisions, will 
probably be an important training requirement with all of the on- and off-board 
data sources. The design of JSF will probably achieve the point that the pilots are 
no longer challenged to collect information, but to use that vast amount of data. 

Another innovative application in the cockpit is reported to be the 3-D 
sound system. “Three-dimensional audio algorithms, to direct appropriate audible 
cues 360 degrees around the pilot are expected to be part of the CNI suite's 
future growth” (Jensen, 2005, para. 33). This feature will apparently reduce the 
visual workload in some cases, and increase SA easily during high workload 
situations. Distribution of signals or information to visual or auditory displays (or 
both), establishing proper standard operational procedures, and training for pilots 
will be necessary to enable them to benefit from this new capability. 

D. POTENTIAL RESEARCH CONCERNS ABOUT HUMAN FACTORS IN 

JSF 

Based on the research literature review and considering the features of 
JSF, the following questions are the major areas of research this thesis tries to 
identify: 
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Concern 1. What are the potential areas that could cause Negative 
Transfer of Training (NTT) problems during transition period? 

Concern 2. How will the pilots filter the data and focus only to the required 

ones? 

Concern 3. What kind of new skills will be needed for JSF? 

Concern 4. What kind of concerns can pose problems about pilot-autopilot 
interaction in JSF? 

Concern 5. Which systems of JSF are going to cause problems to the 
pilots, and thus need to be addressed carefully during training? 
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III. METHODOLOGY 


A. INITIAL SURVEY AND INTERVIEWS 

The main objective of this thesis is to identify and point out the critical 
components in transition, to follow the phases of JSF in terms of Human Factors 
considerations, and then to discuss the possible ways to avoid any problems 
beforehand. 

The initial reflex was to focus on the accidents and mishaps that were 
suffered during and after the previous transition phases of any type of aircraft, 
and then to start building up from them. By this way, the authors planned to 
figure out the common transition concerns, design a flight scenario that would 
cover the most possible human errors along with a task analysis of that particular 
mission scenario, and then run a simulator or part-task trainer-based experiment 
to observe the human error tendencies of pilots from various flight backgrounds. 
These steps could provide to the authors enough bases to analyze the common 
errors and finally offer training guides considering the different aircraft types 
before JSF. 

Throughout the thesis research, the authors realized that there is no 
specific transition study available in public sources that would be directly useful 
for the thesis, but only negative transfer of training studies, which is a dominating 
factor in transition accidents. While brainstorming about how to put these ideas 
together most efficiently, the authors had a chance to make a field trip to 
Lockheed Martin in Fort Worth, Texas. The preliminary survey held during this 
trip of the pilots experiencing the JSF mission simulator has totally changed the 
insight about this study, which will be explained in detail later on. 

The authors think that explaining the reasons and background of this 
change itself will provide a broad perspective about the major Human Factors 
considerations and shifts of the required skills in JSF. 
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This chapter mentions the initial thesis plan; discusses the field trip to the 
Lockheed Martin facilities in Fort Worth for preliminary surveys, interviews, and 
personal experience in an unclassified simulator; and explains the modified plan 
with the rationale behind it. 

1. Initial Plan 

The initial plan of this thesis was to seek NTT issues that may be 
encountered during the transition period. The main human factors concern was 
considered to be problems during transition period, and no other areas were 
thought to pose problems afterward. But the trip to Lockheed Martin changed the 
strategy, and the authors felt impelled revise the thesis strategy. This chapter 
explains the process of this revision throughout this research. 

a. Negative Transfer of Training 

Accidents expedite investigation and research on causes and 
solutions. Accidents also motivate pilots in training to understand the 
circumstances and to avoid them in the future. 

Repetition is one of the key components of learning. If a particular 

procedure or action is followed regularly for a period, it turns into an automated 

response and the human brain takes care of that procedure without the need to 

pay attention to that process. This phenomenon can be explained by the 

following example. In the first few weeks of driving of a car, one needs to check 

the locations of the controls of some systems, such as air conditioning, turn 

signals, windows, etc. After some time, the operator gets used to their locations 

and can control them even without double-checking. This is because our brain 

matches that location with that particular system, and one does not need to direct 

his attention especially to these small operational issues. He just sends signals to 

his brain that he wants to switch the lanes and take the left one, his body takes 

the necessary action automatically and operates the signal handle even without 

thinking. Now imagine that he changes his car, and it has a different user 

interface in which the signal handle works the opposite way. In the beginning, 
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when he wants to switch lanes, he will find himself giving the signal to wrong 
direction, thus causing some confusion for him, and probably unsafe situations 
for the traffic flow. This “negative transfer of training (NTT)” is a common 
phenomenon. Boldovici (1987) explains it as “Practicing Task A interferes with 
learning or performing Task B” (p. 239). When imagining this scenario on the 
ground, it does not sound that critical. However, when things require a quicker 
response, this kind of confusion might pose a life threatening safety problem. 
This is why pilots are prone to safety problems in transition to a new type of 
aircraft more than at any other time. Experiences and automated reflexes are 
hard to unlearn and replace with new ones. Aviation is a very demanding task 
both physically and mentally. Therefore, pilots are required to follow certain 
procedures, usually in very short time periods. This poses a very insidious 
danger for pilots, especially for those who have gained strong habits after many 
flight hours in another type. As pilots acquired many automated skills, this 
becomes a concern especially during transition phases. Apparently, it is hard to 
learn the new operational procedures that conflict with experience. 

This phenomenon formed the basis of the initial thesis plan: to 
identify and point out the possible areas prone to NTT. The major expectation 
was to identify switches operating in an adverse way or critical switches in a 
different location, and those operational or display concepts conflicting with 
experiences depending on particular flight experience. For instance, the so-called 
“Jettison” switch in JSF could be in the same location as the “Master Arm” switch 
of a previous type. This could lead to problems such as an inadvertent jettison of 
the stores while trying to arm them. Another instance could be symbology 
conventions that operate in a different way to indicate the state of the aircraft or 
some avionic systems. 

b. Task Analysis 

Besides identifying the switches or displays that might cause 
problems, the authors also considered the operational procedures required in 
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typical mission scenarios. Air Force officials provided some secret documents 
about JSF that included how to operate the avionics and other Pilot Vehicle 
Interfaces. Those documents are meant to be used for a task analysis in order to 
capture the differences between the JSF and current types. The next step of the 
task analysis plan was to identify the potentially problematic mission types that 
may pose greater risk to pilots, and then conduct the analysis on those missions. 
The opinions from the field trip and the information provided by the documents 
changed this plan. The major reason preventing the task analysis was the 
documents; they were more like technical manuals than operating procedures 
with checklist items. Thus, they could not provide enough background to support 
a robust task analysis. 

c. Experiment 

An experiment was an important part of the initial plan to support 
the thesis with statistical results. The authors planned to focus on some research 
questions after evaluating the preliminary survey results, and analyzing pilot 
interface differences of JSF and previous types. The next step was to devise an 
experiment scenario, probably similar to the one used in task analysis, which 
could enable the authors to cover most of the possible issues. The steps above 
would give the basis for experimental questions and once the access to either 
PC trainer or unclassified simulator was given, experiments could be conducted 
for those questions. 

d. Security Ciassification and Required Literature 

The JSF project is currently engaging in flight and systems tests, 
and even though there is no major change presumed, there are still some 
developments especially in the Pilot Vehicle Interface (PVI) of the aircraft. This 
fact brings commercial security issues along with military security ones. All the 
documents about JSF require a very high level of security clearance. As the 
authors of this study, our advisors and we had this clearance; however, this is a 
major issue for the post-thesis as well as experimentation period. Considering 
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the entities that could make use of or evaluate this study, security would be a 
major problem and most of them would not even be able to read it. So, at some 
point a compromise had to be made in either the scope of study, or the benefits 
that could be gained from it. Another issue would be giving the participants’ 
access to the classified documents or devices during the experiment. 

e. The Problems Experienced 

The first problem, as mentioned before, was the scarcity of the 
literature on studying the transition periods. The reason could be the security 
issues of the military. In addition to the lack of a similar transition study, there 
were also relatively few studies on negative transfer of training in aviation in 
general. Available resources also included U.S. military reports, yet there were 
problems in finding appropriate studies for this thesis. 

The second and most significant problem was the security issue. 
Even if the authors could experience the unclassified Pilot Vehicle Interface (PVI) 
unit, the security measures prevented them from using any training or 
demonstration device of JSF for any further research and experiment efforts on 
this equipment. Therefore, an experiment devised by the authors would not be 
highly related to the PVI of JSF. 

In order to conduct a task analysis robustly, one has to have 
enough resources relevant to the operational procedures and enough insight to 
the context. In this case, the authors have flight experience with the F-16 and its 
operational procedures. Even if they did not have any flight experience with JSF, 
access to the operational procedures of the chosen mission scenarios could 
provide acceptable task analysis on JSF. 

Typical checklists provide sequential procedures about how to 
operate systems on board, as well as emergency procedures (e.g., “Battery - 
ON, Check - Battery ON light”), and these procedures are to be followed during 
the related tasks. Flowever, this information alone is not enough for a task 
analysis, because checklist information covers how to operate systems and the 

61 



actions to be taken in case of problems. It does not cover the interaction 
between the pilot and switches or displays. The document known as “-1” 
(Operating Manual) among aviators is needed for this purpose. The documents 
provided in this case were more technical documents than an operational 
manual. They had technical information that would be useful to explain how to 
operate individual systems; however, there was hardly any information to enable 
one to understand JSF sequential operational procedures. Conducting a task 
analysis with these resources could lead to wrong conclusions. For these 
reasons, the initial thesis plan was revised, which will be discussed in the 
following chapters. 

2. Preliminary Survey 

Ten fighter pilots were assigned to fly the JSF simulator. Pilots had flight 
experience at different levels, both on flight hours and aircraft types. They were 
assigned to fly various missions, both Air-to-Air and Air-to-Ground, and they 
employed various weapons accordingly. They flew the simulator missions from 
the beginning of tactical scenarios, and stopped by the end of each scenario. 
They skipped ground procedures, take-off and landing phases, and the 
navigation phases. Even if these are major phases to be analyzed for a transition 
study, the authors think that the available mission phases were still sufficient to 
support the objective of this thesis. This gave a perfect opportunity to determine 
the PVI issues of the JSF cockpit, at first hand. 

The opinions of the pilots were highly valuable for two reasons. First, 
these pilots have the same kind of flight training and experience as those who 
will fly JSF in the very near future, and some of them probably will do so. So, 
what they experienced in the simulator will not be much different than what will 
be experienced in the future, and most probably there will be even more issues 
due to additional tasks. Second, this was the first JSF simulator experience for 
most of the ten pilots. Therefore, this was a relatively similar first exposure 
scenario even if they did not receive all ground training that the actual JSF pilots 
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will. The lack of ground training could be beneficial for the study, because the 
possible NTT issues would be more apparent. Another point was that the 
participants came to the study from flying experience with varying fighter 
platforms, so the need was anticipated to figure out the differences and 
similarities between pilots of various types, and also their transition suitability to 
this new platform. 

To validate whether the study’s initial plan was on the right track, the 
authors prepared a preliminary survey with open-ended questions. The main 
objective was to have a general understanding of problematic areas and to 
validate the approach of this study. The questions were related to general human 
factors issues and did not require any answers with security classification. 

The preliminary survey questions were as follows: 

1. What is your first impression about JSF? 

2. What are the specific strengths and weaknesses of JSF? (Pilot cockpit 
interaction) 

3. What are the similarities/differences of JSF from your current type of 
aircraft? 

4. Where do you think the potential areas are that might be difficult for a 
transition pilot from your current type of aircraft? 

5. Generally, is the data represented to the pilot at a sufficient level, or 
did it happen that you became overwhelmed by over¬ 
representation/finding and filtering necessary information? 

6. Considering your current aircraft type, is there any system in JSF that 
serves the same purpose with different operation principles/data 
representation/interface/interaction? 

7. Comparing the JSF cockpit with your current type, is there any 
interface/switch that looks similar, and/or is in the same location, but 
used for another system/purpose? 

8. What are your impressions about the specifications of FIMD? (Weight, 
dimensions. Field of View, contrast and brightness range and settings) 
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9. Does the HMD take your focus/attention from priority issues and 
distract you from prioritized task? 

10. Do you have any previous HUD/HMD experience? If so, are there any 
differences? (data representation locations, style and colors) 

11. What are your impressions about the usage and switching of HMD 
modes and its data representation? 

12. What is your general impression about HMD? What are its specific 
strengths and weaknesses? 

13. Do you have any previous MFD experience? If so, are there any 
differences? (data representation locations, style and colors) 

14. What are your impressions about the usage and switching of MFD 
modes and its data representation? 

15. Does the increased symbology confuse/overwhelm you? 

16. What is your general impression about MFD? What are its specific 
strengths and weaknesses? 

17. What is your impression about seat/stick/throttle positioning and their 
usage? 

18. What is your impression about switch positioning on throttle/stick? Is 
there any switch that results in confusion or contradicts your previous 
experience? 

19. What is your impression about 3D audio in operational usage? 

20. Are there any issues regarding the order sequence of operational 
procedures between the JSF and your current type? 

21. In which type of missions/flight phases, is the workload of the pilot 
increased? 

22. What are the effects of increased automation on your workload, SA 
and flight concentration? 

23. What would you like it to be changed in cockpit, and how? 

24. What do you think a pilot in your position would have to learn and/or 
unlearn to fly JSF? 
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As they are straightforward, the questions only require general human 
factors and PVI issues, but not any specific information about any systems. The 
main intent is to capture the potential threats or problematic areas for various 
aircraft types along with any NTT issues. The preliminary survey was handed to 
the participants after they had approximately six missions in the simulator, and 
still had at least this much more to fly, so that they could focus on human factors 
issues with the consideration of survey questions. 

3. Interviews 

It is always a possibility that researchers might miss some important 
points in their survey questions. That was the main reason it was decided to 
interview the participants in addition to the preliminary survey. The advantage 
was that the authors also are fighter pilots and have the same language and 
background with participants. This helped a lot while capturing unmentioned 
points in the survey, and enabled coverage of broader areas. 

Considering the flexibility of a mutual conversation, and the opportunity to 
interview all the participants, the authors went over the questions one by one. 
Noting the process, they could go deeper into the issues that each participant 
brought to the table. The authors mentioned their own experience with the 
unclassified PVI device to the participants, and discussed their experience on the 
same issues. Since everyone had a different approach to answering the survey 
questions, the authors also crosschecked the answers given by other participants 
to validate answers as much as possible. In conclusion, the interviews supported 
the findings from the survey, enabled the authors to go deeper on many subjects, 
and provided more valid results. 

4. The Unclassified Simulator 

The unclassified simulator had the full cockpit interfaces. It was restricted 
on any weapon employment procedures, but capable of all other flight tasks. The 
authors flew the simulator approximately 30 minutes each, and performed take¬ 
off, basic navigation, acrobatic maneuvers, operation of the portals (4 MFD 
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Windows) on the primary LCD display, and landing. They did not have an 
opportunity to experience the revolutionary Helmet Mounted Display (HMD) and 
related systems that work accordingly with it such as Distributed Aperture 
System. 

B. FOLLOW-ON SURVEY 

1. General Information 

The trip to Lockheed Martin in Fort Worth, TX changed the track of this 
thesis substantially. Both the initial and revised thesis plans are thoroughly 
discussed in the related chapter, but the major point is that there was a need to 
identify the human factors concerns further and in more detail before proceeding 
with the scientific process of proposing solutions and testing them. This need 
was the basis for the follow-on survey that will be discussed in this chapter. As 
the initial survey consisted of open-ended questions, and the interviews yielded 
important but general results, the authors decided to construct a survey based on 
their experience from the trip and the literature in a more quantifiable way. The 
following sections will explain the follow-on survey in more detail. 

a. The Structure of the Follow-on Survey 

The objectives of the following survey were different than the initial 

survey. The main objective of the follow-on survey was to identify the possible 

human factors concerns in JSF. The reason to conduct a second survey was 

twofold. First, the initial trip with the survey, interviews and self-experience in 

unclassified simulator yielded different results than the authors predicted the 

human factors issues in JSF would be. And secondly, whatever results the initial 

survey yielded, they were in an open-ended format. Many issues emerged from 

the interviews and initial survey in totally unquantifiable ways. Thus, the 

objectives of the follow-on survey were to confirm the areas that emerged 

strongly in the initial survey, to further investigate the areas that emerged weakly, 

to get the opinions of the participants to solve the possible problems and still 

provide participants some open-ended questions related to both problem 
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identification and solutions. One of the important points is the structure itself. The 
initial survey consisted of totally open-ended answers, whereas the follow-on 
survey benefited from a Likert scale in seven levels. This process helped the 
authors to understand the agreement levels of the participants more accurately 
than the initial survey. 

Another important factor for the follow-on survey was to capture all 
of the areas, systems, or parts of the missions from a human factors perspective. 
For this reason, the survey is divided into five segments. 

In the first segment, the questions try to capture and confirm the 
important points about controlling the aircraft and the possible use of autopilot in 
JSF. Whether or not the basic manual flight skills will be also important in JSF, 
the usability, operation and expected usage of the autopilot are the areas the 
questions investigate. 

Two of the apparent changes or differences of JSF from previous 
generations are its modern and capable systems, and the unique display suite for 
pilot vehicle interfaces. And those areas are questioned in the second segment 
of the survey. Whether it is hard to learn how to operate the systems and menus, 
the systems needing higher workloads that are prone to cause disorientation and 
SA related problems, and which are prone to fixation were among the areas 
investigated in this segment of the survey. 

The third segment is inspired by the prediction that the JSF will 
mainly require mental skills and cognitive resources due to high information load 
provided to pilots and its improved overall capabilities provided by new and 
improved systems. The possibility of informational overload, data filtering issues, 
task overload and workload concerns, demand for mental resources and abilities 
and other attentional areas are investigated in this segment. 

It is commonly accepted that the flight mission begins with the 
preparations before flight, and ends with the debriefing. The more capable an 
airplane is, the more preparation time, briefing and debriefing time it needs. 
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Because there are more capabilities and more sophisticated systems, there must 
be better preparation, more detailed briefings and further coordination among the 
formation members. In order to predict the concerns about the mission planning 
and briefings, the fourth segment of the survey contains questions about the 
possible use of simulators and desktop trainers. 

It is inevitable to miss some of the factors during the surveys, and 
that is the reason for including a segment with open-ended questions. The fifth 
segment of the follow-on survey attempts to learn pilots’ opinions about many 
important issues of JSF from a human factors perspective. The most important 
safety and training concerns the participants foresight, the prediction for the 
transition phase, whether tactical experience of the pilots was important for the 
transition phase, and becoming combat-ready faster in JSF; these are among the 
areas the segment investigated. Likert Style Survey items were used, in which 
participants indicated agreement with these statements from disagree to agree. 

Follow-on survey questions were constructed as follows: 

Question 1: There will be much more use of “autopilot” in the JSF 
compared to my current aircraft type. 

Question 2: No matter how good the autopilot is, pilots will still need 
to train basic flying skills as much as previous type. 

Question 3: Use of the JSF autopilot will greatly help pilots to focus 
on the tactical situation. 

Question 4: The autopilot and other cockpit automation will result in 
possible loss of situational awareness regarding the state of the aircraft position 
control and flight status. 

Question 5; The various flight operating modes in the JSF are easy 
to learn and distinguish. 

Question 6: It is easy to switch between the modes of autopilot and 
transit from autopilot to manual flying. 
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Question 7: The pilot can easily capture any problems of autopilot 
(awareness of aviate & navigate the A/C) when accomplishing other tactical 
tasks in the cockpit. 

Question 8: Because the autopilot of JSF is highly sophisticated 
and has various modes, basic flying skills are not required as much as it is 
required at previous types of aircrafts. 

Question 9: Based on my experience in the JSF simulator, I believe 
that the flight management system is easy to set up and operate. 

Question 10: There are some modes in the flight management 
system that I found difficult to use. 

Question 77; There were instances that I encountered when flying 
the JSF simulator for which I did not understand how to activate or use the 
appropriate operating mode. 

Question 12: As far as I can tell there should be no difficulty 
learning how to configure the cockpit displays for flying, navigating and 
communicating. 

Question 13: It may take extra training time for pilots to learn how to 
effectively operate the new JSF cockpit controls and displays. 

Question 14: There had been instances that I had to focus my 
attention mostly to head-down displays to manage the systems and reach the 
information I needed. 

Question 15: Even if there is a lot of information from various 
sensors on the same display, I did not have any difficulty to filter and evaluate the 
data for decision-making. 

Question 16: DAS can cause disorientation under some conditions. 

Question 77; The idea of seeing HMD symbology wherever I look 
did not distract my attention. 
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Question 18: I believe the HMD failure will dramatically affect the 
mission efficiency. 

Question 19: Managing the switches on throttle and stick effectively 
will require a considerable amount of experience and training. 

Question 20: The appeal of the head-down displays and the 
workload need to be done on those displays might cause flight safety issues. 

Question 21: I believe that there needs to be special training to 
teach pilots how to use the expanded display suite. 

Question 22: Without proper training and experience pilots may not 
be able to handle the vast amount of information provided by the JSF system. 

Question 23: I believe the main task of the pilot will switch from 
mostly flying the aircraft to making tactical decisions. 

Question 24: Being able to follow the whole tactical arena did not 
affect my focus on my own target/area of interest. 

Question 25: Managing both A/A and A/G data at the same time will 
overload pilots under some tactical situations. 

Question 26: I felt the need to effectively filter and declutter the 
presented information in most tactical situations. 

Question 27: Even if JSF presents a very good tactical picture, a 
high level of tactical experience is required to be able to use the capabilities of 
the aircraft to the utmost extent. 

Question 28: Compared to my current type of aircraft, the training 
period should be longer to comprehend the systems thoroughly and fly the 
aircraft at its capabilities. 

Question 29: The new concept of JSF requires building and 
maintaining better SA and more cognitive workload than my current type of 
aircraft. 
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Question 30: There were some instances where I had difficulties at 
shifting my attention between the overall tactical picture and my task related 
tactical picture. 

Question 31 :\ believe a longer pre-flight preparation is needed for 

JSF. 


Question 32: Even if the systems enhance in-flight mutual support 
at a great level, formation briefing and coordination are even more critical than 
for previous types. 


Question 33; The simulator flights and real flights should be exactly 
similar in terms of briefing, mission and debriefing. 


Question 34: PC trainers donated with real throttle and stick 
controls would be significantly beneficial to improve the systems management 
skills of pilots. 


Question 35: To improve the pilots’ display suite management and 
tactical picture assessment skills, alternative-training systems on the ground will 
be helpful other than actual flight conditions. 


Open-ended questions are constructed as follows: 


Question 1: What do you foresee as the most significant problems 
or training issues? Briefly describe. 


Question 2: Which one of the following pilot types do you think will 
qualify to effectively and safely fly all the missions with JSF earlier in transition 
phase; a pilot who gained experience in another aircraft type, or a new graduate 
pilot from flight school? Why? 


Question 3: What might be the most likely cause for flight safety 
problems in JSF? 


Question 4: What would be your recommendations about the 
transition and training phase of JSF? 


Question 5: Other Comments 
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b. Participants 

The participants of the initial and follow-on surveys, as well as the 
interviews have strengths in terms of scientific approach, but some other points 
are the drawbacks that limit the power of the results. 

There were same 10 participants for both the initial and follow-on 
surveys (N=10). Eight of them are current F-16 pilots, and two of them are 
current F-4 pilots. All of them have both Air-to-Air and Air-to-Ground experience, 
and the major positive side for the participants is that they were representative 
for the sake of the study. All of them have the common flight backgrounds and 
training that the JSF pilots will have in the Turkish Air Force. They are all fighter 
pilots from various aircraft types flown in TuAF and are qualified to perform as 
mission commanders. 

If the sample size included many more participants from various 
types of fighter aircraft, it would be possible to compare the results depending on 
the aircraft origins. This could give important clues for each aircraft type. It is 
possible that pilots of one aircraft type could think an issue as a possible 
problem, whereas other type’s pilots wouldn’t. 

The last limiting factor is that the pilots were all from the same 
country. The JSF will be flown in many countries, and because the pilots’ training 
and abilities may vary depending on the countries, participants from other 
countries could provide a broader spectrum for human factors under 
investigation. 

The reason for not having a larger sample size was the security 
and accessibility issues of the JSF project. JSF is under development and has 
commercial security issues as well as military security concerns. 

It is also necessary to mention the experiences of the participants 
with JSF. Obviously, the actual pilots transitioning to JSF will undergo very 
demanding ground training phases before they step into the actual aircraft, but 
this was not the case with participants. They were provided with PVI documents 
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required to operate the necessary systems in the cockpit and had a couple of 
weeks to study them. It is legitimate to say that the participants are not at the 
same experience and knowledge level that the actual transition pilots will be, but 
in contrast they were also not experiencing the actual, real missions. It is to be 
expected that the real missions will pose higher levels of risk/stressors, and will 
be much more demanding than the simulator missions they flew. 

The second important issue about the participants on JSF 
experience is the missions they flew in the simulators. They flew the classified 
mission simulator for both Air-to-Air and Air-to-Ground missions for two weeks. 
The positive side of their experience can be seen in that it was their first 
exposure to the JSF, as it will be for the first transition pilots in the future; thus, 
their opinions are important for providing first glance input. In the second week, 
they also had little more experience than a first glance exposure, and provided 
helpful feedback. But again there is the important factor of actual versus 
simulator missions. The actual aircraft will have full cockpit capabilities once 
released to the air forces, and the missions will be higher risk, and much more 
stressful and demanding. 

Even though there were several shortcomings due to the restricted 
sample, the authors believe that the results are reliable enough for initial problem 
identification and human factors concerns investigation. 
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IV. RESULTS AND DISCUSSION 


A. PRELIMINARY SURVEY WITH INTERVIEW RESULTS 

1. Findings from Interviews, Preiiminary Survey and Own 
Simuiator Experience 

Overall, the iterative answers to both the preliminary survey and the 
interview topics revealed the following outcomes for each question. 

Briefly, the survey and interview results indicated that the most significant 
issue will not be the negative transfer of training, contrary to what was expected. 
The major issues about the transition to JSF appear to be adaptation to new 
technologies on combined flight displays; ability to evaluate, comprehend and 
use the vast amount of information collected from various sensors covering all 
the aircraft; and operating the automated systems. The findings showed that the 
challenge for the pilots is going to be the increased mental workload compared to 
previous aircraft. 


a. Summary of Preliminary Survey Findings 

The following answers were selected from the preliminary survey 
and interviews and are the most agreed upon ones about particular questions, or 
sometimes the interesting ones. For convenience, the authors did not write down 
the entire answers one by one, but rather summarized them by common 
explanations as much as possible. 

1. What is your first impression about JSF? 

Almost all of the participants expressed their first impression that 
they were not expecting such a modern and capable fighter. The most significant 
important input about this question was that the JSF cockpit and PVI were highly 
adaptable. The authors were expecting to get some answers that could be 
attributed to negative transfer of training, yet there was no answer to indicate 
that. One of the strongest comments about the JSF cockpit interface was that it 
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will require a lot more mental workload and situational awareness to be able to 
use all the information presented to the pilots. There were many occasions where 
the participants commented that there might be SA related accidents, mishaps 
and losses due to overwhelming task load and frustration. 

2. What are the specific strengths and weaknesses of JSF (Pilot 
cockpit interaction)? 

The common opinion about this question was that the capabilities 
of the aircraft were both its strengths and its weaknesses. They were the 
strengths as they improve the mission effectiveness dramatically, yet 
weaknesses in that they require considerable amounts of mental effort to operate 
compared to previous types. Obtaining information is not a challenge as before, 
but filtering, evaluating and making decisions while flying the jet were seen as a 
potential hazard for an inexperienced pilot. Most of the participants believe that 
the use of the autopilot will be required in some conditions due to excessive 
mental workload. 

3. What are the similarities/differences of JSF from your current type 
of aircraft? 

The most obvious agreement among the pilots about the difference 
of JSF from their current types was that JSF required more mental effort. The 
pilots having relatively less modern cockpits found the logic in operating the 
modern systems easy, and others who already operate a glass cockpit similar to 
JSF found the operational procedures to be similar or if different, easy to adapt. 

4. Where do you think the potential areas are that might be difficult for 
a transition pilot from your current type of aircraft? 

The participants considered managing all information provided in 
the cockpit to be a major challenge. They thought that this issue needs to be 
addressed carefully during the initial phases of training via all training devices 
including the simulator. 
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5. Generally, is the data represented to the pilot at sufficient level, or 
did it happen that you became overwhelmed by over¬ 
representation/finding and filtering necessary information? 

There is more information being presented to pilots than ever 
before, but the problem is to manage that much information. 

6. Considering your current aircraft type, is there any system in JSF 
that serves the same purpose with different operation 
principles/data representation/interface/interaction? 

No significant issue reported about this question. 

7. Comparing the JSF cockpit with your current type, is there any 
interface/switch that looks similar, and/or is in the same location, 
but used for another system/purpose? 

No significant issue reported about this question. 

8. What are your impressions about the specifications of FIMD 
(Weight, dimensions. Field of View, contrast and brightness range 
and settings)? 

It is accepted as a very useful system helping pilots to build up their 
SA. The more experienced pilots consider the HMD and DAS combined usage 
as a potential threat due to some conditions that might lead to disorientation, and 
thus result in an undesirable event. 

9. Does the HMD take your focus/attention from priority issues and 
distract you from prioritized tasks? 

In general, the HMD symbology did not prevent participants from 
focusing on events outside the cockpit, but while “seeing through” the fuselage 
and through the other cockpit displays, some participants found it disorienting 
initially, and commented about being extra cautious to use this system especially 
at low altitude flights and in bad weather conditions. 
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10. What are your impressions about the usage and switching of HMD 
modes and its data representation? 

No significant issue reported about this question. 

11 .What are your impressions about the usage and switching of HMD 
modes and its data representation? 

No significant issue reported about this question. Only some pilots 
not familiar to HMD reported that they prefer to use Head Down instruments to 
reach information due to their habits. However, they also stated that it is a time 
based issue and can be overcome in short time. 

12. What is your general impression about HMD? What are its specific 
strengths and weaknesses? 

The general conclusion about the HMD was its usefulness for pilots 
to enhance their mission effectiveness. 

13. Do you have any previous MFD experience? If so, are there any 
differences (data representation locations, style and colors)? 

The participants operating MFDs in their current types reported that 
the operational logic and the interface of similar systems in the JSF cockpit are 
totally different, thus posing no NTT issue. 

14. What are your impressions about the usage and switching of MFD 
modes and its data representation? 

The portals (Separate MFD Windows of the big display suite) are 
easy to adapt and use. However, it is stated that lacking a thorough training on 
modes and symbology will prevent pilots from obtaining the necessary 
information when needed. 

15. Does the increased symbology confuse/overwhelm you? 

The portals provide perfect SA, and were found to be very useful, 
but the existence of a lot of data and modes was considered as a major 
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challenge during operations, and also a means of distraction due to containing 
too much information at the same time; thus, it was seen as crucial to declutter 
the irrelevant information. 

16. What is your general impression about MFD? What are its specific 
strengths and weaknesses? 

General opinion was that the display suite enhances the pilot’s SA 
and is very powerful as well as being useful. 

17. What is your impression about seat/stick/throttle positioning and 
their usage? 

The anthropometric design of the throttle, stick and seat, felt 
comfortable. There were pilots who used to fly with a more straight seat and stick 
in the middle of cockpit, as well as pilots who were familiar with the side stick and 
HOTAS switchology with slightly more seat angle to the back, yet none 
complained about the seat and throttle-stick setup. 

18. What is your impression about switch positioning on throttle/stick? 
Is there any switch that results in confusion or contradicts your 
previous experience? 

The participants using HOTAS in their current types report the 
HOTAS in JSF as easily adaptable. Even if there were switches at the same 
location or the same purpose but which operated differently, they found it very 
easy to adapt. They expressed the importance of its familiarization and training 
for an effective use because it controls even more systems than it ever has 
before. 

19. What is your impression about 3D audio in operational usage? 

The 3d audio system was not used. 

20. Are there any issues regarding the order sequence of operational 
procedures between the JSF and your current type? 
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The participants stated that they did not have enough knowledge 
and information about the operational procedures of JSF to compare with their 
experience. The tasks they were supposed to perform while on JSF simulator 
flights covered limited operational knowledge about the JSF platform. 

21. In which type of missions/flight phases, is the workload of the pilot 
increased? 

The pilots reported that workload was especially high in low altitude 
missions, which require a lot more tasks to accomplish, yet do not allow pilots to 
use autopilot to concentrate on tactical tasks. 

22. What are the effects of increased automation on your workload, SA 
and flight concentration? 

Autopilot is considered to be very useful. Considering the high task 
load to maintain tactical awareness and effectively use the weapons, pilots stated 
that its use is necessary. Some participants even stated it is compulsory to use 
autopilot to effectively make best use of the aircraft systems. 

23. What would you like to be changed in cockpit, and how? 

No significant issue reported about human factors in cockpit. 

24. What do you think a pilot in your position would have to learn 
and/or unlearn to fly the JSF? 

Some answers include the need for a robust brain-muscle 
coordination, getting used to cockpit instruments and displays, and maybe 
starting all over again except general aviation knowledge and skills. These are 
also considered as the basic skills to fly any aircraft. Flowever, besides the 
physical demands of flying an aircraft, JSF requires a highly adaptable mind for 
new technology, being able to filter and evaluate a large amount of information. 
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B. ANALYSIS OF THE FOLLOW-ON SURVEY RESULTS 


1. Multiple Choice Questions 

Results from the multiple-choice questions are shown in Figures 5 through 
39. The histograms in these figures indicate the agreement level of participants 
with the related survey questions. The Y-axis indicates the number of participants 
for the given agreement levels. The sum of the numbers in some questions is not 
ten despite the fact that there were ten participants. This is because some 
participants stated that they had no opinion about those related questions; thus, 
they are not represented in these histograms. Considering that all the histograms 
are very self-explanatory, they are not named specifically. 

The major point of the analysis is the nature of the data or answers. One 
could think to assign numbers between -3, and 3 to represent a value range from 
“strongly disagree” to “strongly agree,” and then calculate the average for each 
question. For instance, a typical report would be “The average agreement level 
for this question is 2.3.” But as the answers are categorical (ordinal), it would be 
wrong to apply any linear mathematical calculation to them. The moderate 
agreement level assigned the number 2 would not be two times stronger than the 
agreement level of “slightly agree” with the number 1. This is the reason for 
reporting the agreement levels in frequency histograms. The histograms provide 
the tendencies of the participants in each question graphically and numerically. 
After reporting each question’s histograms without any conclusion, the discussion 
section of this chapter will provide the summary and discussions of the results, 
and report the important findings of the follow-on survey. 
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Question 1: There will be much more use of “autopilot” in the JSF 
compared to my current aircraft type. 

Question 1 
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Figure 5. Answers to Question 1 

Question 2: No matter how good the autopilot is, pilots will still need to 
train basic flying skills as much as previous type. 
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Figure 6. Answers to Question 2 
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Question 3: Use of the JSF autopilot will greatly help pilots to focus on the 
tactical situation. 


Question 3 
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Figure 7. Answers to Question 3 

Question 4: The autopilot and other cockpit automation will result in 
possible loss of situational awareness regarding the state of the aircraft position 
control and flight status. 


Question 4 
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Figure 8. Answers to Question 4 
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Question 5: The various flight operating modes in the JSF are easy to 
learn and distinguish. 
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Question 5 
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Figure 9. Answers to Question 5 


Question 6: It is easy to switch between the modes of autopilot and transit 
from autopilot to manual flying. 


Question 6 
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Figure 10. Answers to Question 6 
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Question 7: The pilot can easily capture any problems of autopilot 
(awareness of aviate & navigate the A/C) when accomplishing other tactical 
tasks in the cockpit. 

Question 7 
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Figure 11. Answers to Question 7 

Question 8: Because the autopilot of JSF is highly sophisticated and has 
various modes, basic flying skills are not required as much as it is required at 
previous types of aircrafts. 





Question 9: Based on my experience in the JSF simulator, I believe that 
the flight management system is easy to set up and operate. 
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Question 9 
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Figure 13. Answers to Question 9 

Question 70; There are some modes in the flight management system that 
I found difficult to use. 



Question 10 
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Figure 14. Answers to Question 10 
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Question 1 7;There were instances that I encountered when flying the JSF 
simulator for which I did not understand how to activate or use the appropriate 
operating mode. 


Question 11 
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Figure 15. Answers to Question 11 

Question 12: As far as I can tell there should be no difficulty learning how 
to configure the cockpit displays for flying, navigating and communicating. 
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Figure 16. Answers to Question 12 
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Question 13: It may take extra training time for pilots to learn how to 
effectively operate the new JSF cockpit controls and displays. 



Figure 17. Answers to Question 13 

Question 14: There had been instances that I had to focus my attention 
mostly to head-down displays to manage the systems and reach the information I 
needed. 



Figure 18. Answers to Question 14 
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Question 15: Even if there is a lot of information from various sensors on 
the same display, I did not have any difficulty to filter and evaluate the data for 
decision making. 
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Figure 19. Answers to Question 15 
Question 75; DAS can cause disorientation under some conditions. 
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Figure 20. Answers to Question 16 
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Question 77; The idea of seeing HMD symbology wherever I look did not 
distract my attention. 


10 

9 

8 

7 

6 

5 

4 

3 

2 

1 

0 


Question 17 




strongly Moderately Slightly Neiital Slightly Moderately Strongly 

Disagree Disagree Disagree Agree Agree Agree 


Figure 21. Answers to Question 17 

Question 18 :1 believe the HMD failure will dramatically affect the mission 
efficiency. 
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Figure 22. Answers to Question 18 
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Question 19: Managing the switches on throttle and stick effectively will 
require a considerable amount of experience and training. 



Figure 23. Answers to Question 19 

Question 20: The appeal of the head-down displays and the workload 
need to be done on those displays might cause flight safety issues. 


Question 20 


i.i 

Slightly Neutal Slightly Moderately Strongly 

Disagree Agree Agree Agree 


Figure 24. Answers to Question 20 
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Question 21: \ believe that there needs to be special training to teach 
pilots how to use the expanded display suite. 



Figure 25. Answers to Question 21 

Question 22: Without proper training and experience pilots may not be 
able to handle the vast amount of information provided by the JSF system. 



Figure 26. Answers to Question 22 
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Question 23: I believe the main task of the pilot will switch from mostly 
flying the aircraft to making tactical decisions. 
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Figure 27. Answers to Question 23 

Question 24: Being able to follow the whole tactical arena did not affect my 
focus on my own target/area of interest. 
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Figure 28. Answers to Question 24 
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Question 25: Managing both A/A and A/G data at the same time will 
overload pilots under some tactical situations. 

Question 25 
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Figure 29. Answers to Question 25 

Question 26 :1 felt the need to effectively filter and declutter the presented 
information in most tactical situations. 


Question 26 
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Figure 30. Answers to Question 26 
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Question 27: Even if JSF presents a very good tactical picture, a high level 
of tactical experience is required to be able to use the capabilities of the aircraft 
to the utmost extent. 


Question 27 
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Figure 31. Answers to Question 27 

Question 28: Compared to my current type of aircraft, the training period 
should be longer to comprehend the systems thoroughly and fly the aircraft at its 
capabilities. 
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Figure 32. Answers to Question 28 
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Question 29: The new concept of JSF requires building and maintaining 
better SA and more cognitive workload than my current type of aircraft. 



Figure 33. Answers to Question 29 

Question 30; There were some instances where I had difficulties at shifting 
my attention between the overall tactical picture and my task related tactical 
picture. 



Figure 34. Answers to Question 30 
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Question 31 :\ believe a longer pre-flight preparation is needed for JSF. 


Question 31 
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Figure 35. Answers to Question 31 

Question 32: Even if the systems enhance in-flight mutual support at a 
great level, formation briefing and coordination are even more critical than for 
previous types. 



Figure 36. Answers to Question 32 
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Question 33; The simulator flights and real flights should be exactly similar 
in terms of briefing, mission and debriefing. 



Figure 37. Answers to Question 33 

Question 34: PC trainers donated with real throttle and stick controls 
would be significantly beneficial to improve the systems management skills of 
pilots. 



Figure 38. Answers to Question 34 
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Question 35; To improve the pilots’ display suite management and tactical 
picture assessment skills, alternative-training systems on the ground will be 
helpful other than actual flight conditions. 
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Question 35 
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Figure 39. Answers to Question 35 


2. Open-ended Questions 

It is always possible to miss important points in the survey questions. 
Another possibility is that there are additional aspects that the participants can 
provide to the argument. Those were the reasons to include the open-ended 
questions to the follow-on survey, to allow participants to add their opinions and 
additional comments about transition, training and safety concerns of JSF. 

The results are provided again as summaries based on the common 
answers to each question, as well as some less frequent but interesting ones. 
Five questions were asked, and the results can be seen below. 

Question 1: What do you foresee as the most significant problems or 
training issues? Briefly describe. 

Many participants saw the capabilities of JSF as being significantly better 

than the current types, and that there will be more demand for technical and 
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tactical knowledge to fly JSF. The JSF will require more resources in training due 
to its increased capabilities; these will include such things as wider training 
areas, ground training devices, and many more. Another important common 
opinion is the importance of the tactical experience, and its currency. The 
majority of the participants agreed that the transitioning pilots should have 
tactical experience before JSF in order to fly safer and more efficiently due to its 
dominating capabilities, and the abundant information it provides. 

Question 2: Which one of the following pilot types do you think will qualify 
to effectively and safely fly all the missions with JSF earlier in transition phase: a 
pilot who gained experience in another aircraft type, or a new graduate pilot from 
flight school? Why? 

There is a solid consensus about this issue, and the participants think that 
the JSF is relatively intuitive and easy to adapt. It also provides all and maybe 
more than necessary information to the pilots, with its numerous systems, but 
they add that it will need experience to filter and evaluate the alternatives, in 
order to make quick and robust decisions. Thus, they prefer experienced pilots to 
be sent to JSF. 

Question 3: What might be the most likely cause for flight safety problems 
in JSF? 

Two agreements emerge from this question. The first one is related to 
HMD and DAS related disorientation hazards. The pilots think that disorientation 
is a potential threat, and suggest that operating procedures should be 
established properly and pilots well trained in those systems’ use. The second 
concern is the display suite with the portals. The participants report that 
especially in high workload conditions, fixating on the display suite may be a very 
common and risky concern about flight safety. As almost all of the mission 
related tactical information is presented on that display, there will be a tendency 
for the pilots to fixate on it, and be unaware or less aware of other parameters 
and outside information. 
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Question 4: What would be your recommendations about the transition 
and training phase of JSF? 

The participants see two factors as important in terms of training and 
transition to JSF. They want the training period to be very intensive, well planned, 
and also to have ground training devices. The main reason for this kind of 
intensive training is the extensive capabilities of the systems. The more capable 
systems require more dynamic training. Another comment the participants make 
is the importance of employing experienced pilots in the transition periods first, to 
be followed later by the less experienced pilots. 

Question 5: Other Comments 

No important comments were made for this question. 

C. DISCUSSION OF THE FINDINGS 

This section is intended to summarize the major points that emerged from 
the follow-on survey. Detailed answers to the questions can be found in the 
previous section of this thesis. 

This discussion is based on comments and responses by the participants 
and the opinions of the authors. The discussion of the findings is divided into 
three topics: cockpit automation, safety concerns, and training transition. This 
classification is based on the major concerns raised from the answers, but also 
they reflect the literature review chapter of the thesis. The major arguments 
about modern cockpits from a human factors perspective also contain the same 
topics. 

As is thoroughly discussed in the related work chapter, automation has 
had dramatic impacts on the human user’s role and responsibilities; the more 
modern, and complex systems in the cockpits required higher levels of cognitive 
skills, higher workloads, and higher training requirements. This section will not 
provide the related literature; for further information the reader can refer to 
Chapters II and IV. 
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1. Cockpit Automation 

As discussed in Chapter IV, automation is used effectively in commercial 
aviation, but not yet in military aviation, especially in fighter cockpits. Although 
many individual systems (such as radars) had automatic features, the autopilot 
has been rarely used in the military. Especially in dynamic situations, military 
pilots prefer to have manual control of the aircraft. All these factors have reduced 
the human factors problems in fighter cockpits so far. But the question of whether 
this will change in JSF is raised after reviewing the design philosophy of the 
aircraft. That question required further investigation and was the reason the 
survey included many questions regarding this concern. 

The major finding about this issue is that the participants think the 
autopilot will be used more than in their current aircraft. Additionally, they agree 
that the main task of the pilot will shift from actually flying the aircraft to making 
tactical decisions. These findings show a need to think about the impacts of 
automation in the JSF cockpit. The problems and hazards found in the literature 
will probably also be a concern in fighter cockpits beginning with JSF. 
Considering the riskier, more dynamic and higher workload of fighter missions, 
there is a need to understand which problems will also be an issue in fighter 
cockpits and to what extent. 

After acknowledging that the autopilot will have more use in the JSF 
cockpit, the participants add that it will not mean that the basic flying skills will no 
longer be necessary. The handling and control capabilities are expected still to 
be important skills. 

Another concern raised by the participants is to understand if the autopilot 
in JSF is considered easy to learn and operate by the pilots, and if the autopilot 
may cause the pilots to lower their awareness about the task it performs. The 
main opinion is that it will be easy to learn and operate, and to be aware of the 
modes of autopilot. At the same time, some participants believe that the use of 
autopilot may decrease the awareness of the pilots of the parameters it 
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manages. But they also think that the pilots will be able to overcome any 
problems about automation or the parameters it controls, too. In conclusion, 
automated flight features may be easy to learn and operate, but even if there is 
slight agreement on the decrease in awareness, it is expected that the pilots will 
be able to overcome any related problems. Considering that the real missions will 
be more stressful and demanding, the authors think that these concerns about 
cockpit automation require further research for more precise understanding, and 
may pose a higher threat than indicated by the participants’ expectations. 

The last agreement about automation is that it does what is claimed in the 
JSF cockpit. The participants agree that the autopilot and other automated 
systems helped them to focus more on the tactical picture. Other important 
aspects of this finding will be discussed in the next section. 

2. Tactical Decision Making and Systems Management 

This section contains a wide spectrum of human factors areas, such as 
SA, workload, spatial disorientation, and cognitive tunneling. Many question were 
asked about the use of the systems, workload, safety traps, and other potential 
issues. 

One of the objectives of the survey was to capture the opinions of the 
participants about the usability of the cockpit controls and displays. The concern 
here is not to capture the workload and stress during the missions, but whether it 
is easy to operate the systems, such as changing modes, making modifications 
and managing the menus. There is a general agreement that the operation of the 
flight management systems is easy. The participants did not find that there were 
modes that were hard to understand. The participants indicated that it will be 
easy to learn how to operate those systems and that the management system 
will probably not be a major problem. 

There was no general consensus about whether it will take extra time to 
learn to effectively operate the controls. However, when the question is related to 
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the control stick and throttle, and special emphasis is given to the display suite, 
the participants think that they should be addressed very carefully and focus on 
using all necessary training equipment. 

Both ample information from various sensors, and possible tactical 
changes in air combat could present higher workloads to pilots. Additionally, the 
big LCD suite will be the main display while performing tactics in those high 
workload situations. If the autopilot is also used in these situations, awareness, 
cognitive tunneling, and other related hazards might be a potential threat. To 
investigate these types of threats, the survey included two groups of questions. 

The first group of questions addressed the concern that the participants 
had times when they needed to focus, or fixate on the display suite. The opinion 
is that there is a need for using the display suite in many tactical situations. The 
participants agreed that there is a reliance on the display suite in general too. It is 
legitimate to say that there is no doubt in the participants’ answers, that the pilots 
will be using the display suite very frequently in many tactical situations. 

After assuming the fact that the pilots will focus mostly on the display 
suite, the following question is whether it will pose a threat to flight safety. And 
figuring out this issue was the objective of the second mentioned group of 
questions. Some of the related answers are that there were instances when the 
pilots think they focused on the display suite more than they think is safe, and 
they generally think there is a risk that the pilots will fixate on this display suite 
and become unaware of other information such as “fuel state”, “altitude 
limitations” and many more. Especially in higher workload situations, the 
participants report that they needed to focus more on the displays in order to 
reach and evaluate the tactical information. 

According to the participants, the threat mentioned above is obvious, but 
with the more specific questions about the display suite, there are blurred areas. 
The authors expected to find data filtering difficulties, but the participants 
reported that even if there was a great deal of information, they did not have 
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difficulties in reacting appropriately. Similarly, no significant agreement is 
reported about the existence of both Air-to-Air and Air-to-Ground symbology at 
the same time. Roughly, half of the participants report that it may overload the 
pilots, and the remaining half do not agree. Additionally, they also did not agree 
clearly on the need to declutter some symbology. 

Another controversial issue is whether the pilots will have attentional focus 
problems due to the fact that they will be presented with a very big picture of the 
battle area. For instance, a flight leader may be responsible to intercept an 
enemy formation, but in JSF he will have almost all battlefield information in front 
of him, such as other friendly assets and both air and ground threats. After a 
while, he should focus on building intercept geometry, target sorting, and 
monitoring the target formation in more detail. Whether other “big picture” 
information will distract him from focusing on his target is an issue. Will the pilot 
be able to focus his attention back and forth between his targets and the general 
picture? The participants’ answers reveal that they did not have any difficulty in 
focusing on their target areas; however, half of them experienced instances 
where they had difficulties in shifting their attention between the overall tactical 
picture and their task related picture. 

Surprisingly, the participants did not have any major difficulty in filtering 
the data, managing both Air-to-Air and Air-to-Ground data simultaneously, and 
operating the display suite. Additionally, they did not report any apparent 
problems in shifting attentional focus, and they expressed that it was easy to set 
up and operate the systems. Flowever, when asked whether a high level of 
tactical experience is required to use the capabilities of JSF to the utmost extent, 
the participants answered in the affirmative. They had many comments 
addressing the importance of the tactical experience. The participants agree that 
filtering and evaluating the data, as well as making quick and proper decisions 
will require tactical experience. The reason for the contrary aspect can be 
twofold. Either the participants found the use of the display suite, attentional 
focus, and data filtering not to be a problem, because they could cope with many 
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issues, given their experience, or they see the complex cognitive processes while 
decision making and managing the displays, and processing information, as 
separate. And they think that while the former task is hard, the second tasks are 
relatively easier. Either way, the experience factor cannot be denied, but the data 
filtering and display suite operation in actual missions require further research. 

Disorientation and distraction were also the concerns of this thesis. Both 
are related to the visual systems, especially HMD and DAS, as the authors 
predicted. The participants do not support the first suspicion of the authors, that 
the HMD will distract the pilots’ attention with the symbology it presents 
continuously. They report that seeing HMD symbology wherever they look did not 
distract their attention. But the major potential hazard expected by the 
participants is the DAS, if the pilot is not trained to use it properly. The 
participants agree that the DAS may cause disorientation in some conditions. 
Additionally, most of them mentioned the same risk in their open-ended answers. 
They also commented that the training and operational procedures of this system 
should be established very carefully. 

3. Transition Training 

Among the areas investigated in the survey, the concerns related to 
training during the transition period showed the strongest agreement among the 
participants. The survey explored the opinions of the participants about the 
training in two major topics. In the first one, the systems’ training issues are 
questioned, whereas in the other one, the training system is investigated as a 
whole. 

The systems of interest in this context are HOTAS, the display suite, and 
DAS. The survey has explicit questions for the first two systems, and the 
recommendations about DAS training came mostly from the answers related to 
safety concerns. The reports about DAS are mentioned in the previous section. 
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and the emerged agreement is that it may be prone to cause disorientation in 
some conditions, and properly should be addressed during training to mitigate 
any potential hazard. 

The initial thought of the authors was that pilots would need more 
emphasis on training for HOTAS management. HOTAS is not new to the cockpits 
and pilots, but the reason is that the JSF has more systems with enhanced 
capabilities, and thus more switches and menus in HOTAS. Additionally, the 
pilots may have a higher workload in order to use all capabilities of such an 
advanced fighter. However, the general opinion of the participants is neutral on 
the question of whether HOTAS management will demand considerable amounts 
of experience and training, but they mostly agreed that it will be very useful to 
have a PC-Trainer type of device with real HOTAS to train pilots about the 
management of the system. When it comes to the training of the display suite, 
there is a stronger agreement. The participants mostly agree that special 
emphasis should be given to display suite management during training. 
Additionally, evaluating the information from the displays for tactical assessment 
was also found to be an important training issue, and participants strongly agreed 
that alternative training devices such as PC-Trainers will be very beneficial for 
this purpose. Another supporting fact for the opinion of using alternative training 
devices is that the participants also mentioned it in their open-ended answers. 
Addressing the same cognitive processes while pilots make complex decisions in 
very demanding situations during ground training looks to be more important in 
JSF than in the current types. 

Another aspect of interest about the training is the general opinions of the 
participants. The most general question related to this issue is whether JSF will 
require a longer training period than the current aircraft to make pilots combat 
ready. Whereas the general tendency was to agree this statement, three out of 
ten pilots disagreed. One of the critical factors of the participants’ JSF 
background is that they did not fly missions according to any performance 
grading criteria, but just experienced various missions and gave feedback. Both 

107 



during actual JSF training, exercise and combat missions, the pilots will have real 
objectives that will be measured objectively. It is highly probably that the more 
demanding and risky missions could give the participants a better perspective, 
and they could be able to see the difficulty levels about the tasks better, and 
more realistically. 

While slightly agreeing that JSF should have a longer training period, the 
participants strongly agree that longer preflight preparation will be needed for 
JSF missions. Moreover, they also strongly agree that even if the systems 
enhance mutual support to a large degree, formation briefing and coordination 
will be even more critical than in current fighters. Another agreement is that the 
participants think that the simulator missions should be as similar as possible to 
actual missions. All of the aforementioned strong agreements mean that the 
participants think the missions in JSF will be more demanding, and thus require 
longer preparation, more coordination, briefing and more simulator practices. 
Altogether, these comments indicate a need for longer and more demanding 
training in JSF. 

An interesting point emerged among the answers, in that the participants 
also agreed about the role shift phenomenon. Some of them commented that the 
training should focus more on data filtering and decision-making in complex 
situations skills. These skills are also important for the current types, but with the 
increasing amount of data, enhanced capabilities, and possibly harder demands 
during JSF missions, these skills may become more important than before, and 
should be specifically addressed during the curriculum. 

The participants agreed strongly about several aspects of the transition 

period: The training devices such as simulators and PC-Trainers should be used 

to the utmost extent; transition pilots should have tactical experience before JSF. 

The main reason is that the participants think the experience is the crucial factor 

for evaluating the data, and using it for making tactical decisions in JSF. In order 

to cope with abundant data, and high workload, the participants think experience 

is critical. Especially, the consensus is that the first group of transition pilots 
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should be very carefully selected from among the experienced pilots, and two 
participants think that the less experienced pilots could be also transitioned 
directly to JSF after the first group, but with a well researched, and developed 
instructional curriculum. The participants agree that the pilots should be trained 
thoroughly for each difficulty during ground training to prepare them to 
demanding situations during actual missions. The issues include tactical 
decision-making, using both A-A, and A-G capabilities simultaneously, 
information reaching and evaluation capabilities, and switchology. 
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V. CONCLUSIONS AND RECOMMENDATIONS 


A. SUMMARY OF KEY FINDINGS 

Although there are more concerns that emerged from the research of this 
thesis, this chapter will focus on the ones most agreed upon by the participants. 
Some of these predictions are also strongly supported by the literature. Findings 
about automation and display suite operations are examples in this category. 
Other findings cannot be directly found in the literature, but not because they do 
not have scientific support. The main reason is that they are related to the new 
systems in the cockpits such as DAS. Although there are many studies about 
HUD and HMD, the DAS has many unique capabilities that prevent using the 
findings of HUD and HMD to solve the potential problems in DAS. The potential 
concerns are given in a summary sentence format; for more detail, the previous 
chapter will provide enough information. 

1. Increasing Use of Autopilot 

The need to use the autopilot at a significantly increasing level is one of 
the findings of the follow-on survey. The participants also acknowledged the role 
shift of the pilot due to increased automation. Even though participants did not 
directly point out a problem related to mode or system awareness during 
simulator missions, the authors believe that the findings in the literature about 
increasing autopilot use will be valid and applicable to JSF. Most of the studies in 
the literature examined autopilot use in commercial aircraft. Even though the 
tasks and required pilot skills vary from commercial cockpits to fighter cockpits, 
the common human factors issues also will be experienced in the JSF cockpit. 
The JSF pilot will have more tasks at hand, that are to be accomplished in a 
much more stressful environment, and with greater demand on cognitive 
resources; thus, the authors predict that JSF pilots will also be prone to 
automation-related human factors concerns. 
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2. HMD and DAS Usage 

Throughout aviation history, it has been frequently observed that while 
technology and inventions solve the current problems, they also give rise to new 
human factors concerns. DAS represents a totally new concept in the fighter 
cockpit, and the participants strongly agree that it may cause disorientation in 
some cases. They think that the operational procedures and DAS usage in 
different environmental and meteorological conditions should be thoroughly 
examined and addressed in pilot training. 

3. Display Suite Management 

The display suite in JSF is the most noticeable component of the cockpit. 
This display will be the prominent PVI element during flights, where the pilot can 
reach all the sensor data and interact with the systems as required. The features 
of the previous aircraft displays are combined into this display in JSF, thus 
making the display more appealing and prone to visual and cognitive tunneling 
issues as seen in the literature. The participants also agree that this display suite 
may cause cognitive tunneling, and fixation issues during operations. 

4. Tactical Experience and Decision Making 

The participants strongly agreed that tactical experience is one of the most 
vital features for a pilot to make the most out of the systems in JSF. Even though 
filtering the data from various sensors is not thought to be difficult by the 
participants, assessing this data in order to make demanding tactical decisions 
will require significant tactical experience. Considering the changing mission 
concept of JSF, combined Air-to-Air and Air-to-Ground data, and new systems, 
the authors agree with participants on the importance of tactical experience. 
Dealing with more systems and data with the same attentional resources will be 
one of the crucial aspects of the required skills in JSF. The participants not only 
stated the importance of tactical experience in regular missions, but also during 
transition and follow-on training phases. 
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B. TRAINING OBJECTIVES 


After identification of the major concerns, there is a need to address them 
during training and while establishing operational procedures of the aircraft. 
These two processes are closely related; if there is an operational procedure, the 
system trains the trainees to accomplish that procedure. This section includes 
the training objectives that are thought to mitigate any potential problem related 
to the identified concerns. The training objectives are created to meet the 
common problems found by the literature, and predicted by this study. The 
training objectives do not cover all of the required objectives related to each 
predicted concern, but just the major findings of this thesis. 

1. Increasing Use of Autopilot 

• The pilots should have a thorough understanding of the automation 
technical data, operation and modes, and their interactions with 
other systems. 

• The pilots should perceive automation as another important system 
of the aircraft and build skills for robust coordination and operation. 

• The pilots should be aware of the tasks performed by autopilot and 
autopilot modes continuously, even under high workload conditions. 

• The pilots should also be able to solve the emergency situations 
related to automation malfunctions and while automation controls 
the parameters. 

2. HMD and DAS Usage 

• The pilots should build robust mental models of HMD and DAS and 
their operational procedures. 
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The pilots should be ready to accomplish their tasks during HMD 
failure successfully and they should cross check the parameters 
shown in HMD with Head-Down indications as required. They 
should not over trust the HMD. 

The pilots should learn to transition safely back and forth between 
Head-Up and Head-Down operations. In addition to Head-Up to 
Head-Down transitions, the DAS transitions are predicted to be 
critical phases of the flight. The pilots will “see through” the 
fuselage and look for targets of interest under the aircraft, etc., and 
then they will have to transition back to the normal vision or vice 
versa. Problems such as disorientation and attentional shift are 
predicted, especially when these transitions happen during high 
workload, adverse weather, and other demanding situations. The 
pilots should be able to use, and transition to or from the DAS 
safely. Another aspect of this issue is the need for well-defined 
operational procedures that will enable the pilots to comprehend 
the current situation, and make proper assessments depending on 
the operational procedures. 

Display Suite Management 

The pilots should acquire full knowledge about the display suite. 
The raw technical information, modes, and menus of the portals, 
meanings of the symbology, and the required skills for operational 
procedures are among the important aspects related to the display 
suite management. 

The pilots should be able to setup, operate and modify the portals 
effectively according to the mission requirements. 

The pilots should cross check the data shown with various sensors 

if possible, and be able to anticipate any problems about the 

symbology and presented information. 
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• The pilots should be ready to accomplish their tasks successfully 
during display or portal failure. 

• The pilots should not fixate on the display even under high 
workload tactical situations, and should also be aware of other 
parameters. 

• The pilots should build comprehensive knowledge about the 
interactions and cooperation among the systems feeding the 
display suite. In case of a failure of any of those systems, the pilots 
should be aware of the reliability of the information at hand and 
make correct decisions in these contingency situations. 

4. Tactical Experience and Decision Making 

• The pilots should be proficient enough to filter the unnecessary 
data easily, and to locate the required information in the tactical 
displays. 

• The pilots should be able to process and evaluate the presented 
data quickly and accurately. 

• The pilots should have proper prioritization and attention allocation 
skills and techniques for highly demanding and high workload 
situations. 

• The capable and modern sensors will flood the pilots with a great 
deal of information, and it is likely that the pilots will experience 
many situations that are complex, hard to analyze, and in which it is 
hard to make proper decisions. The pilots should be capable of 
making quick and accurate decisions under such complex 
scenarios. 
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C. GENERAL TRAINING GUIDE 

The predictions of this thesis are parallel to the research literature. 
Important aspects of modern glass cockpit aircraft are generally similar in terms 
of the human factors perspective. Increasing automation, LCD or MFD displays, 
increasing sensors with enhanced capabilities—all present new challenges to 
pilots. Thus, the authors think that the solutions proposed in the literature will 
also be applicable to the JSF case. The limitations explained in previous 
chapters prevented conduct of actual experiments; therefore, the basis for this 
chapter is the research literature, participants’ comments and the experience of 
the authors. 

1. Increasing Use of Autopilot 

Objective 1.1 The pilots should have a thorough understanding of the 
automation technical data, operation and modes, and its interactions with other 
systems. 

Objective 1.2 The pilots should perceive automation as another important 
system of the aircraft, and build skills for robust coordination and operation. 

One of the findings supporting the importance of these objectives is that 
Carter (2000) reports that pilots do not have a through understanding of the 
structure and operational procedures of automated systems. 

Rigner and Dekker (2000) are among the researchers proposing 
incorporation into pilot training of a curriculum dedicated to automation. They 
note that automation training is given in transition training today, and shifting that 
to earlier fundamental training may be a better approach. 

Rigner and Dekker (2000) also observed that the content of the current 
automation training is incomplete. The training is more based on general 
technical information about the automated systems and very limited scenarios. 
They claimed that automation should be introduced and taught as a team 
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member of the cockpit rather than a subsystem of the cockpit systems. This is 
the case in most of the phases of flight, and autopilots make decisions and give 
inputs as the pilots do. 

Casner (2003) claimed that almost all current aircraft benefit from 
automation, and that this topic can therefore be addressed in basic flight training. 
This stage is where pilot trainees get their introduction to general topics about 
aviation such as weather, aerodynamics, flight rules and many others. They are 
not assigned to their final aircraft type yet, and usually fly their first aircraft in their 
careers. However, this is a suitable time to train them about the fundamentals of 
cockpit automation. Currently, the automation training varies from type to type. 

To study the subject further, Casner (2003) conducted an experiment to 
measure the effectiveness of such a curriculum. In addition to a control group, 
the target group received the aforementioned training in a small aircraft cockpit, 
and the target group demonstrated positive training transfer in their second trials 
in a commercial aircraft’s simulator while performing automation-related tasks. 

The fighter pilots scheduled to fly JSF should be trained on the 

fundamentals of automated systems, and especially autopilots in their initial pilot 

training, and then build their professional skills with their aircraft-specific type 

trainings. Where this solution is not applicable, this training can be implemented 

within the JSF training itself. Either the fundamentals and basic principals may be 

given before proceeding with the training of automated systems of JSF, or they 

may be directly integrated to JSF autopilot systems training. At a first glance, the 

second way sounds more practical, but further research is required in order to 

figure out the better way scientifically. The objective of this change is to provide 

pilots better background about cockpit automation, and thus help them build 

more robust mental models about all automated systems in the JSF cockpits. 

During this training phase, another important objective should be to help pilots 

build robust perception about the autopilot. Considering that the pilots may use 

the autopilot in demanding situations, it is crucial to give them the required skills 

for effective cooperation. The pilots should be directed to not perceive the 
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autopilot as a “fire and forget” system, but rather as a separate system to be 
cooperated with, and checked for proper operation when necessary. The 
recommended topics for the automation ground training for JSF pilots are as 
follows: 

Recommended Initial Automation Training Topics 

• The Basic Structure and Principals of Autopilot Systems 

• Common Problems Found by Literature Concerning Cockpit Automation 

• The Interactions and Interrelations between the Autopilot and Other 

Systems in JSF 

• System Analysis and Problem Solving of Automation in JSF 

• Analysis of the Autopilot Modes in JSF and Potential Threats During 

Missions 

• Pilot-Autopilot Cooperation Procedures 

Objective 1.3 The pilots should be aware of the tasks performed by 
autopilot and autopilot modes continuously, even under high workload conditions. 

Objective 1.4 The pilots should be able to solve the emergency situations 
related to automation malfunctions and while automation controls the 
parameters. 

The findings of Carter and Woods (1994) supported the common fact 
about pilots on automation. They report that pilots were comfortable in 
performing the standard basic tasks in cockpits such as “intercepting a radial, 
building or executing a holding pattern...,” but they experienced difficulties during 
the tasks requiring comprehensive understanding about automation such as 
“aborting a takeoff at 40 kts with autothrottles on” and “...anticipating when go- 
around mode becomes armed through landing...” (p. 14). 
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Similarly, the findings of Sarter, Mumaw, and Wickens (2007) were also 
consistent with the previous studies. During their research, both with subjective 
and objective measures, the pilots allocated considerably more of their 
attentional resources on basic flight parameters than on the automated systems. 
Their automation awareness was much lower than their general awareness, and 
one of the reasons was reported as improper mental models about the 
automation. 

There are two major points related to Objectives 3 and 4: the awareness 
problem, and the lack of skills to solve complex situations. A broader discussion 
about the decision-making skills in complex situations will be given in the Tactical 
Experience section of this chapter, and the emergency situation solving skills 
about the automation are the major concern about the automation training 
objectives. 

The recommendations concerning the above issues are as follows: 

Increasing the Awareness and Enhancing The Complex Problem Solving 
Skills of Pilots about Cockpit Automation 

• Develop the Operational Procedures so that they make sure the pilots also 
control the automation-related parameters and modes as they do with 
other flight parameters. 

• Ensure that pilots acquire those skills mentioned above during ground 
training, especially in simulator missions; set sortie objectives related to 
the awareness of automation in JSF cockpit. 

• In simulator missions, stress the emergency solving capabilities of pilots in 
two conditions: the autopilot is controlling some parameters, and any other 
system failed, causing an emergency situations, or again the autopilot is 
controlling some parameters, and it itself fails causing an emergency 
situation. Inject these kinds of events to the simulator missions also under 
high workload situations, while pilots are busy with tactical decisions, and 

tend to forget to check what the autopilot is doing. 
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2. HMD and DAS Usage 

The results of both interviews and surveys do not indicate any major 
problem with HMD use. In contrast, the situation changes with DAS and the 
participants agree that DAS operational procedures, and training curriculum 
should be specifically established in order to mitigate any disorientation or 
attentional focus concerns. Even though there have been many studies 
conducted about the HMD and HUD in fighter cockpits, the DAS is a brand new 
system. The authors were not able to locate any studies about DAS use in 
cockpits; thus, the predictions by the participants are the main resource to 
anticipate any problems, as well as to propose solutions. 

The training objectives regarding HMD use mostly address the 
effectiveness issues, but the DAS training objectives are more related to flight 
safety; thus, they are more critical. 

Objective 2.1 The pilots should build robust mental models of HMD and 
DAS, and their operational procedures. 

Because the DAS is a new system, there is no coverage of related 
systems in basic flight training to provide a framework for the JSF pilots’ DAS 
training. Pilots do not need deep knowledge of the technological aspect of DAS, 
but the focus is more on the operational procedures. The operational procedures 
and the safety concerns with DAS should be analyzed in high detail. This is the 
first step by the instructional designers and the frontiers of the JSF before it finds 
its way to air forces. Then, the second important step is the training of the pilots. 
Given the fact that their experiences will not include a system similar to DAS, 
they should be all trained about the anticipated safety issues such as transitions 
back and forth in adverse weather conditions, and the normal procedures. The 
objective with these training topics is to prevent any mishaps that could happen 
because the pilots do not follow the operational procedures or use DAS in 
conditions where they should not use it. They also should be trained about the 
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scientific fundamentals of disorientation or attentional focus problems possible 
with DAS use. Recommendations are as follows: 

The Recommended Topics About Das Ground Training 

• The Capabilities and Limitations of DAS 

• Emphasis on the Standard Operational Procedures 

• Human Factors Concerns and Potential Threats About DAS 

(Disorientation, Attentional Focus) 

• The Use of DAS in Adverse Conditions 

• DAS Emergency and Recovery Procedures 

Objective 2.2 The pilots should be ready to successfully accomplish their 
tasks during HMD failure, and they should cross check the parameters shown in 
HMD with Head-Down indications as required. They should not over trust the 
HMD. 

One of the important projections of this study is the potential effect of HMD 
on mission efficiency. The HMD is predicted to be very helpful to the pilots during 
their missions. Thus, there may be a risk to over trust the HMD and the pilots 
may choose to accomplish their missions using the HMD more than they should. 
There are two crucial points about this concern. The first one is that the pilots 
may not crosscheck HMD indications with the Head-Down displays. This is a 
common error in current fighters. The pilots are instructed to use the Head-Down 
displays as their main displays and crosscheck the HUD or HMDs with Head- 
Down displays while they shift their focus to HUD or HMD. The authors do not 
have the technical information about the HMD system of JSF. Unless the 
designers eliminated any possibility that the HMD can display different 
information than the Head-Down displays, this issue will continue to be crucial in 
terms of efficiency and flight safety. 

The second important aspect is the skills of the pilots. While over-using 

HMD, they may degrade their capabilities to accomplish their missions using 

121 



other displays or systems, and in case of a HMD failure, their efficiency may be 
affected dramatically. Unless the operational procedures will dictate aborting the 
missions in case of HMD failure, this will also be one of the important training 
objectives. Below are the recommendations for these purposes. 

Enhancing the Awareness and Atypical Operational Skills of Pilots about 

HMD 

• Inject Events of Partial HMD or HMD Symbology Failures Requiring a 
Crosscheck With Head Down Instruments To Resolve. 

• Inject events of Total HMD Failure and Train Pilots to Accomplish All 
Mission Types Allowed in SOP. 

Objective 2.3 The pilots should acquire the capability to transition safely 
back and forth between Head-Up and Head-Down operations. 

Other than Head-Up to Head-Down transitions, the DAS transitions also 
are predicted to be critical phases of the flight. The pilots will “see through” the 
fuselage and look for targets of interest under the aircraft, etc., and then they will 
have to transition back to normal vision or vice versa. Problems such as 
disorientation or attentional shift concerns are predicted, while these transitions 
happen during high workload, adverse weather, and many other demanding 
situations. The pilots should be able to use, and transition to or from the DAS 
safely. Another aspect of this issue is well-defined operational procedures that 
will enable the pilots to comprehend the current situation, and make proper 
assessments depending on the operational procedures. 

Preparing the Pilots for DAS Transitions and Use 

• Present demo situations concerning use or non-use of DAS, and observe 
that the pilots make proper decisions of when to use or not use the 
system. 
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• Inject high workload scenarios requiring the use of DAS, and observe that 

the pilots totally follow the SOP and checklist items while using HD and 

DAS. 

• Train pilots for adverse weather and many other demanding situations of 

DAS use.. 

• Train pilots for recoveries after the problems caused by improper DAS 

use, such as disorientation and attentional shift/focus problems. 

3. Display Suite Management 

Objective 3.1 The pilots should acquire full knowledge about the display 
suite. The raw technical information, modes, and menus of the portals, meanings 
of symbology, and the required skills for operational procedures are among the 
important aspects related to the display suite management. 

Objective 3.2 The pilots should be able to setup, operate and modify the 
portals effectively according to the mission requirements. 

Objective 3.3 The pilots should cross check the data shown with various 
sensors if possible, and be able to anticipate any problems about the symbology 
and presented information. 

Objective 3.4 The pilots should be ready to accomplish their tasks 
successfully during display or portal failure. 

Objective 3.5 The pilots should not fixate on the display even under high 
workload tactical situations and should also be aware of other parameters. 

Objective 3.6 The pilots should build comprehensive knowledge about the 
interactions and cooperation among the systems feeding the display suite. In 
case of a failure of any of those systems, the pilots should be aware of the 
reliability of the information at hand and make correct decisions in these 
contingency situations. 
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The aim of the training objectives of this section is the technological 
knowledge and operational capabilities related to the display suite. How to use it 
in demanding tactical situations and the decision-making processes are 
addressed in the following chapter. There are two driving facts that make the 
display suite a crucial system in JSF. First, it will be the main interface while 
operating JSF systems. Almost all required tactical data are fused and presented 
in this display. And the other point is about a safety concern directly related to the 
aforementioned fact. The participants foresee a potential threat of fixation on the 
display suite during operations. 

Display suite management is also one of the concerns of the design team. 
Adams (2007) made interviews with the design officials in the JSF project and 
they confirmed that the display suite as one large LCD screen instead of the 
older display systems is one of the important innovations in the JSF cockpit. The 
need of addressing how to manage this display suite is also one of the training 
objectives of the design team, and a PC Trainer system is developed for this 
purpose. 

If pilots can increase their proficiency and speed in managing the display 
suite, it will free their valuable cognitive resources during high workload 
situations. They will be able to focus on the decision-making processes and how 
to acquire the important data quickly, but not how to operate the display 
management system. Below are the recommendations for the objectives related 
to display suite management. 

• Train the pilots in display suite management until they become proficient, 
even under high workload conditions while operating the display suite. 

• Make sure that the pilots totally understand the menus and the symbology 
of the portals. 

• Train to setup the display suite properly depending on the dynamic 
mission requirements, and ask for mission critical data in high 
informational overload and high workload conditions. 
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• Train the pilots to crosscheck the presented data from redundant sensors. 

• Train pilots for display malfunctions and make sure that they are capable 

of accomplishing the allowed mission types under these abnormal 

conditions. 

4. Tactical Experience and Decision Making 

Objective 4.1 The pilots should be proficient enough to filter the 
unnecessary data easily, and obtain the required information from the tactical 
displays. 

Objective 4.2 The pilots should be able to process and evaluate the 
presented data quickly and properly. 

Objective 4.3. The pilots should have proper prioritization and attention 
allocation skills, and techniques for highly demanding and high workload 
situations. 

Objective 4.4 The modern sensors will overload the pilots with too much 
information, and it is likely that the pilots will experience many situations that are 
complex and hard to analyze, and for which it will be hard to make proper 
decisions. The pilots should have the capability to make quick and proper 
decisions under such complex scenarios. 

Objective 4.1 is mainly covered in the previous section, but it is also 
related to this section in terms of experience. It also takes experience to know 
where to look in order to find data. Expert pilots know where to look and when, 
and easily access required information with less effort. The prioritization 
capabilities, filtering and evaluating the data, attentional allocation, and decision 
making under high informational and stress load are all interrelated. This is the 
reason to keep and analyze all of the objectives together to propose solutions. 

The important finding of the FAA research, as perceived by these authors, 
about flight training that is reviewed in Chapter III is that shifting the curricula to a 
more SBT orientation, with real consequences in order to address the complex 
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decision-making capabilities, has the potential of providing the trainees enhanced 
cognitive capabilities they will require during their demanding tasks in their 
modern cockpits (Fiduccia et al., 2003; French et al., 2005; Robertson et al., 
2006; Dornan et al., 2007). Considering the finding that the participants strongly 
agree on the importance of the tactical experience in JSF, the decision-making 
mechanism of the experts also emerges as crucial for JSF pilots. 

The recommendations about the tactical experience and decision-making 
in JSF are as follows: 

Recommendations for Decision Making and Acquiring Tacticai 
Experience Faster 

• Present pilots with tactical pictures on the ground and ask for specific 
decisions. This can be accomplished both by using static pictures of the 
displays or simulator and desktop trainers. 

• Various demanding tactical pictures along with critical decisions can be 
discussed in classrooms. 

• Inject high workload into the mission scenarios and train pilots on fixation 
issues. 

• Train pilots for proper prioritization and attentional allocation skills 
depending on the mission demands, task at hand, and workload. Inject 
various demanding tactical scenarios to observe proper skills by the 
trainees. 

• The terminology used during the mission communications, how to operate 
the displays, filter, acquire the data, and making decisions are considered 
to be taught via simulators and desktop trainers with FIOTAS; thus, 
missions such as BVR engagements, SEAD, and many more can be 
taught on the ground. The pilots will struggle less with the terminology, 
display management, and decision-making in the air after such training 
with effective debriefings. And their tactical experience will build up earlier. 
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D. CONCLUSIONS 

Throughout the thesis research, the authors always aimed to both identify 
and solve the potential human factors issues in JSF. Although the initial thesis 
plan was built around the potential NTT issues during transition periods only, the 
initial and follow-on surveys did not predict it as an issue. 

The potential concerns identified by this thesis are: the possible problems 
during the use of automation, potential disorientation concerns in DAS use, 
display suite management, and complex cognitive skills required both for tactical 
decision-making and information management processes. 

The research literature review, subjective reports by the participants and 
the operational experience of these authors formed the basis for the 
recommendations on pilot training in JSF. A curriculum revision from the 
traditional training curriculums is recommended to address both the potential 
automation-related problems and to ensure that the pilots build relevant cognitive 
skills, by utilizing the ground training devices as much as possible. 

E. FUTURE RESEARCH RECOMMENDATIONS 

One of the two objectives of this thesis is to identify the human factors 
concerns in JSF, and the other is to develop recommendations for training to 
solve the concerns. In the follow-on survey analysis chapter, the shortcomings of 
the survey were provided, but there remain many other methods for problem 
identification and solution creations. The aim of this chapter is to provide the 
framework for possible future research for both processes. 

1. Problem Identification 

The most important phase of problem solution is to understand the 
situation first. If this step is not totally fulfilled, one cannot propose robust 
solutions. This thesis only had the opportunity to benefit from limited resources 
for a subjective method for problem identification, and what could be done in 
order to have a better subjective research is briefly discussed in Chapter V. 
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a. Research Recommendations 

• In all probability, there are ongoing analyses and surveys at 
Lockheed Martin about usability, workload, and many other 
concerns, as well as their solutions related to human factors 
concerns in JSF. The decision-making skills of the test pilots may 
be captured in highly demanding complex situations, and other than 
establishing SOP, normal, and emergency operational procedures. 
These skills may be imported to JSF training. This thesis both 
supports the ongoing studies and provides new insights on them. 

• JSF will be flown in many allied countries. Each country may have 
differences in terms of pilot training, and subjective studies may be 
conducted in order to identify various concerns depending on the 
countries or types of aircraft. This research will yield different 
results depending on the background; thus, the training curriculums 
can be tailored to specific needs if necessary. 

• The pilots with experience with the JSF simulator, desktop trainer 
or actual aircraft may be also surveyed for their training 
recommendations to identified problems. 

2. Objective Methods 

Another methodology is the objective research. Many experimental 
designs could serve this objective, but the limitations prevented these authors 
from getting access to any JSF trainer or simulator in order to conduct an 
experiment. The design team, and many other frontier officials already are 
involved with this objective, but because of the security issues, the authors do 
not have robust knowledge what has been done. But one important fact is that 
only the design team, JSF program officials, and those researchers having 
access can do these experimental studies. 
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The authors do not have enough knowledge about the capabilities of the 
actual aircraft, full mission simulator, part task trainers, or desktop trainers in 
terms of conducting an experiment; thus, it would not be legitimate to propose an 
experimental design. The recommendations below are only suggested as the 
framework for possible experiments for further research. 

a. Recommendations for Experimental Research 

• All of the findings of this thesis, or any other research related to 
JSF pilot concerns, can be evaluated using various methods, such 
as simulation. 

• For automation findings, scenarios with various mission 
requirements, informational overload, and workload level can be 
injected into controlled scenarios and while accomplishing the 
tasks; the awareness levels of pilots with automated variables can 
be measured. Also, demanding atypical procedures can be injected 
into experimental scenarios to measure the participants’ complex 
decision making skills about automation; thus, it can be determined 
whether their mental models are good enough. The second step 
after these experiments would be testing the solutions. After 
establishing proper operational procedures to address the 
problems, and providing the appropriate training, similar 
experiments would serve also for testing whether the training is 
effective or not. A control group with no training, and other groups 
with other types of training will be beneficial to understanding 
further the effectiveness of the proposed training solutions. 

• For DAS use, the experiments depend on the simulator capabilities. 
The required extent for motion, and visual cues to create situations 
to the real aircraft are crucial in order to conduct an experiment on 
the ground. That has to be validated by the test pilots, or any other 
source as possible. If the simulators are good enough, various 
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scenarios in day and night conditions, and in adverse weather 
conditions with dynamic maneuvers should be constructed where 
the DAS use is either a choice or mandatory. If the situation 
requires a choice, whether the pilots face difficulties as predicted or 
not can be tested. And if the use is mandatory by mission 
requirements, the situations prone to mishaps can be also identified 
objectively. The following step would be to test the training 
solutions. Similar scenarios can be constructed for trained pilots in 
order to measure their performance, making sure that the issues 
are addressed effectively. 

• For display management and tactical decision making capabilities, 
two important questions are whether the pilots could build 
experience with ground training devices effectively or not, and 
whether proposed newer learning curriculums will help pilots with 
their complex decision making skills or not. For the second 
question, a group with similar training as F-16, F-15, or many other 
current type-training curricula can be used, and the JSF group will 
be trained with the proposed curricula. The differences can be 
tested both with decision-making scenarios, where the pilots are 
asked to make decisions to given problems. Informational overload 
also can be evaluated. 

• Other than the decision making process, the effect of training 
devices on tactical experience is also an important question. The 
main idea is that the experts make quick and proper decisions with 
less effort from their experiences. The end-result of this 
phenomenon is that if a pilot makes a decision fast and properly, 
and understands the current tactical situation easily, it can be 
claimed that he has enough tactical capability. Two groups could be 
used to measure this question, one group with proposed 
curriculum, using training devices for tactical scenarios as 
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explained in training recommendations section, and the second 
group with a traditional training curriculum. Both groups’ 
performance can be measured during actual tactical missions, and 
either the elapsed time until a certain goal can be measured, or at a 
certain time the performance differences can be measured. 

It is certain that many of these recommendations have been considered 
and many others are implemented by the officials in the JSF project, but research 
from various sources is helpful to increase confidence, or to gain more insight 
into the situations. Also, many important points are expected to emerge after the 
aircraft begins to fly in the air forces, but the goal for researchers and related 
officials should be to predict actual problems in advance, and solve them prior to 
use. 
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